The Blog of BenningerJekyll2024-02-05T13:56:49-08:00https://benninger.ca/Chrishttps://benninger.ca/chris@benninger.cahttps://benninger.ca/posts/celery-serializer-pydantic2024-02-05T00:00:00-08:002024-02-05T00:00:00-08:00Chrishttps://benninger.cachris@benninger.ca
<h1 id="automatic-serialization-with-celery-and-pydantic-models">Automatic Serialization with Celery and Pydantic Models</h1>
<p>I’m working on a project involving FastApi (which uses Pydantic models under the hood) and Celery. I wanted to have a nice clean way to pass Pydantic models to Celery tasks so I could avoid translating back and forth. To my surprise this doesn’t work natively, though it does seem like <a href="https://github.com/celery/celery/issues/8751">it has been considered</a></p>
<p>I ended up solving this using Celery’s ability to add custom serializers and <a href="https://stackoverflow.com/questions/21631878/celery-is-there-a-way-to-write-custom-json-encoder-decoder">this helpful starting example</a>.</p>
<h3 id="structure">Structure</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>main.py
models/
__init__.py
models.py
pydanticserializer/
__init__.py
pydanticserializer.py
</code></pre></div></div>
<h3 id="modelspy">models.py</h3>
<p>Create a module around this</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kn">from</span> <span class="nn">pydantic</span> <span class="kn">import</span> <span class="n">BaseModel</span>
<span class="k">class</span> <span class="nc">MyCustomTypeOne</span><span class="p">(</span><span class="n">BaseModel</span><span class="p">):</span>
<span class="n">myfield</span><span class="p">:</span><span class="nb">str</span>
<span class="k">class</span> <span class="nc">MyCustomTypeTwo</span><span class="p">(</span><span class="n">BaseModel</span><span class="p">):</span>
<span class="n">myotherfield</span><span class="p">:</span><span class="nb">str</span>
</code></pre></div></div>
<h3 id="pydanticserializerpy">pydanticserializer.py</h3>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="kn">import</span> <span class="nn">json</span>
<span class="kn">from</span> <span class="nn">pydantic</span> <span class="kn">import</span> <span class="n">BaseModel</span>
<span class="kn">import</span> <span class="nn">models</span>
<span class="k">class</span> <span class="nc">PydanticSerializer</span><span class="p">(</span><span class="n">json</span><span class="p">.</span><span class="n">JSONEncoder</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">default</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">BaseModel</span><span class="p">):</span>
<span class="k">return</span> <span class="n">obj</span><span class="p">.</span><span class="n">model_dump</span><span class="p">()</span> <span class="o">|</span> <span class="p">{</span><span class="s">'__type__'</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">obj</span><span class="p">).</span><span class="n">__name__</span><span class="p">}</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">json</span><span class="p">.</span><span class="n">JSONEncoder</span><span class="p">.</span><span class="n">default</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">pydantic_decoder</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>
<span class="k">if</span> <span class="s">'__type__'</span> <span class="ow">in</span> <span class="n">obj</span><span class="p">:</span>
<span class="k">if</span> <span class="n">obj</span><span class="p">[</span><span class="s">'__type__'</span><span class="p">]</span> <span class="ow">in</span> <span class="nb">dir</span><span class="p">(</span><span class="n">models</span><span class="p">):</span>
<span class="n">cls</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">models</span><span class="p">,</span> <span class="n">obj</span><span class="p">[</span><span class="s">'__type__'</span><span class="p">])</span>
<span class="k">return</span> <span class="n">cls</span><span class="p">.</span><span class="n">parse_obj</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="k">return</span> <span class="n">obj</span>
<span class="c1"># Encoder function
</span><span class="k">def</span> <span class="nf">pydantic_dumps</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>
<span class="k">return</span> <span class="n">json</span><span class="p">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">cls</span><span class="o">=</span><span class="n">PydanticSerializer</span><span class="p">)</span>
<span class="c1"># Decoder function
</span><span class="k">def</span> <span class="nf">pydantic_loads</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>
<span class="k">return</span> <span class="n">json</span><span class="p">.</span><span class="n">loads</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">object_hook</span><span class="o">=</span><span class="n">pydantic_decoder</span><span class="p">)</span>
</code></pre></div></div>
<h3 id="mainpy">main.py</h3>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">celery</span> <span class="kn">import</span> <span class="n">Celery</span>
<span class="kn">from</span> <span class="nn">kombu.serialization</span> <span class="kn">import</span> <span class="n">register</span>
<span class="kn">import</span> <span class="nn">pydanticserializer</span>
<span class="c1"># Register new serializer methods into kombu
</span><span class="n">register</span><span class="p">(</span>
<span class="s">"pydantic"</span><span class="p">,</span>
<span class="n">pydanticserializer</span><span class="p">.</span><span class="n">pydantic_dumps</span><span class="p">,</span>
<span class="n">pydanticserializer</span><span class="p">.</span><span class="n">pydantic_loads</span><span class="p">,</span>
<span class="n">content_type</span><span class="o">=</span><span class="s">"application/x-pydantic"</span><span class="p">,</span>
<span class="n">content_encoding</span><span class="o">=</span><span class="s">"utf-8"</span><span class="p">,</span>
<span class="p">)</span>
<span class="n">celery_app</span> <span class="o">=</span> <span class="n">Celery</span><span class="p">()</span>
<span class="n">celery_app</span><span class="p">.</span><span class="n">conf</span><span class="p">.</span><span class="n">update</span><span class="p">(</span>
<span class="n">task_serializer</span><span class="o">=</span><span class="s">"pydantic"</span><span class="p">,</span>
<span class="n">result_serializer</span><span class="o">=</span><span class="s">"pydantic"</span><span class="p">,</span>
<span class="n">event_serializer</span><span class="o">=</span><span class="s">"pydantic"</span><span class="p">,</span>
<span class="n">accept_content</span><span class="o">=</span><span class="p">[</span><span class="s">"application/json"</span><span class="p">,</span> <span class="s">"application/x-pydantic"</span><span class="p">],</span>
<span class="n">result_accept_content</span><span class="o">=</span><span class="p">[</span><span class="s">"application/json"</span><span class="p">,</span> <span class="s">"application/x-pydantic"</span><span class="p">],</span>
<span class="p">)</span>
</code></pre></div></div>
<p>So now that that is wired up you should be able to setup tasks that accept and return these models natively</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">@</span><span class="n">celery_app</span><span class="p">.</span><span class="n">task</span><span class="p">(</span><span class="n">bind</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">mytask</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">req</span><span class="p">:</span> <span class="n">MyCustomTypeOne</span><span class="p">)</span> <span class="o">-></span> <span class="n">MyCustomTypeOne</span><span class="p">:</span>
<span class="p">...</span>
<span class="k">return</span> <span class="n">MyCustomTypeTwo</span><span class="p">(</span><span class="n">myotherfield</span><span class="o">=</span><span class="s">"..."</span><span class="p">)</span>
</code></pre></div></div>
<p>…As well as invoke them in this way</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="n">req</span> <span class="o">=</span> <span class="n">MyCustomTypeOne</span><span class="p">(</span><span class="n">myfield</span><span class="o">=</span><span class="s">"..."</span><span class="p">)</span>
<span class="n">async_result</span> <span class="o">=</span> <span class="n">mytask</span><span class="p">.</span><span class="n">delay</span><span class="p">(</span><span class="nb">input</span><span class="p">)</span>
<span class="p">...</span>
<span class="n">result</span><span class="p">:</span> <span class="n">MyCustomTypeTwo</span> <span class="o">=</span> <span class="n">async_result</span><span class="p">.</span><span class="n">get</span><span class="p">()</span>
</code></pre></div></div>
<p><a href="https://benninger.ca/posts/celery-serializer-pydantic/">Automatic Serialization with Celery and Pydantic Models</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on February 05, 2024.</p>https://benninger.ca/posts/intel-throttling2021-01-27T00:00:00-08:002021-01-27T00:00:00-08:00Chrishttps://benninger.cachris@benninger.ca
<h1 id="finding-a-solution-to-strange-cpu-throttling-on-linux">Finding a solution to strange CPU throttling on Linux</h1>
<p>I have a <code class="language-plaintext highlighter-rouge">Lenovo L590</code>, a mid-tier thinkpad. Sick of 1 hour battery life on my old laptop, I picked the CPU with the lowest power consumption available for this model at the time, the <code class="language-plaintext highlighter-rouge">Intel i5-8265H</code>. The CPU has a 1.6GHz base clock and a 3.4GHz boost clock.</p>
<p>I use it happily for light work, like browsing and development side projects etc. But when Covid hit I was finding myself using Zoom (and other non-hardware accelerated conferencing tools) constantly. I started noticing that the calls would get choppier and slower the longer the call lasted.</p>
<p>After some investigation, I realized that my fan was never coming on and the CPU was getting hot and throttling itself progressively further and further down until it could find a place it could avoid overheating at. To my surprise, I was seeing that my CPU firmware (not the governor) was throttling itself down as far as 800 MHz to keep from overheating, and yet always staying under 70 deg Celcius.</p>
<p>The pattern I was seeing was that the CPU would run at a healthy clock-speed and boost up frequently until it hit exactly 70 degrees, at which point the fan would never engage at any RPM (which I was able to validate using thinkfan) and would begin to aggressively clock down all while staying at exactly 70 degrees under load.</p>
<h1 id="using-thinkfan-to-avoid-downclocking">Using <code class="language-plaintext highlighter-rouge">thinkfan</code> to avoid downclocking</h1>
<p>The first assumption was that Lenovo had setup an embarassingly poor fan-curve and the solution was to force my own fan curve.</p>
<p>I setup <a href="https://github.com/vmatare/thinkfan">thinkfan</a> and began to tinker. The intention was to force a new fan curve so that the fan would keep the CPU cool enough to avoid clocking down.</p>
<p>For benchmarking I used:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sysbench --threads=8 --test=cpu --time=120 run
</code></pre></div></div>
<p>My original sysbench score before any tinkering:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>events per second: 5432.80
</code></pre></div></div>
<p>I did a lot of experimentation with the fan curve and could never get quite right, things still seemed artificially stuck at 70 degrees and the CPU was never seeming to kick into high gear.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>echo "watchdog 0" > /proc/acpi/ibm/fan
echo "level 4" > /proc/acpi/ibm/fan
watch -n 1 cat /proc/acpi/ibm/fan
</code></pre></div></div>
<p>I started to experiment with benchmarking the CPU while having the fan pinned to a high RPM just to see what would happen. I noticed about a 10% improvement in performance from pinning the fan, but It wasn’t what I was hoping for.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>events per second: 6005.07
</code></pre></div></div>
<h1 id="using-throttled-to-solve-the-real-problem">Using <code class="language-plaintext highlighter-rouge">throttled</code> to solve the real problem</h1>
<p>While I was combing through Lenovo forums I stumbled upon <a href="https://github.com/erpalma/throttled">throttled</a></p>
<p>This tool completely unscrewed things for me. Instantly the CPU was spiking up to 3.4GHz under load, passing 70 degrees and the built-in fan curve would kick in and cool it back down. After using this tool, I was able to actually remove thinkfan altogether.</p>
<p>The new sysbench score:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>events per second: 7457.87
</code></pre></div></div>
<p>After seting up <code class="language-plaintext highlighter-rouge">throttled</code> and setting my fan curve back to auto I was able to achieve a 28% sustained benchmark performance. And even more impressively the boost clock actually went above 3.4GHz.</p>
<p>Beyond simple benchmark performance however the more aggressive upper clock and the improved ability to sit at the base clock, the machine is significantly snappier while using the interface and just feels much less sluggish.</p>
<p>If you run Linux on a laptop with a CPU in that rough class I suggest experimenting with <code class="language-plaintext highlighter-rouge">throttled</code></p>
<p><a href="https://benninger.ca/posts/intel-throttling/">Intel CPU Throttling</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on January 27, 2021.</p>https://benninger.ca/posts/fussel2020-09-29T00:00:00-07:002020-09-29T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
<h1 id="fussel-gallery">Fussel Gallery</h1>
<p>I’ve been working to scratch a personal itch for months. The goal was to find a tool to:</p>
<ul>
<li>Display photos</li>
<li>Separate photos by face-tags</li>
<li>Provide a unique URL per face-tagged person</li>
<li>Require absolutely no backend running code (I don’t need another thing to maintain)</li>
</ul>
<p>I spent months searching and found nothing. So I wrote my own tool <a href="https://github.com/cbenning/fussel">Fussel</a></p>
<p><a href="https://benninger.ca/posts/fussel/">Fussel Project</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on September 29, 2020.</p>https://benninger.ca/posts/force-dns-go-through-pihole2019-07-23T00:00:00-07:002019-07-23T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
<h1>The Pi-hole</h1>
I've been having great success with my <a href="https://pi-hole.net/">pi-hole</a>. For those unaware of what pi-hole is, it is a DNS black-hole tailored to run on the <a href="https://www.raspberrypi.org/">rasperry pi</a>.
The way it works is, you set it up as the upstream DNS in your DHCP server (which is usually your router) and then when clients ask for an IP address the router also tells them to use the pi-hole as their upstream DNS.
<br/>
<br/>
So what happens now, is when a machine on your network asks for a domain like "google.com" the pi-hole receives it checks it against its list of malicious and tracker domains and then happily forwards it up to the real upstream DNS server (in my case CloudFlare)
However, when a website tries to pull down some file like "https://someadnetwork.biz/tracker.js", the pihole will receive the DNS lookup for `someadnetwork.biz`, recognize it as a tracking domain and just pretend it cannot find it.
<br/>
<br/>
This will result in a significant reduction in ads you see on most websites and a snappier experience as the amount of traffic is reduced. In my case, I saw a rate of 14% of all DNS queries being blocked by my pi-hole. 14%!
<h1>The Problem</h1>
When you have good quality standard computer hardware, things work out just as outlined above. Proper network devices will honor the settings given to them by the DHCP server and use your pi-hole as their upstream DNS allowing it to filter their DNS requests.
<br/>
<br/>
Unfortunately in the ad-revenue-driven world of subsidized devices from Amazon and Google (to name two of many), the cost of the device is heavily subsidized by the manufacturer by means of collecting, storing and selling the ever-living-sh*t out of everthing
they can. Sadly I am forced to have a few of these devices running on my network and I keep them pretty isolated, however I was noticing that I was still receiving ads on these devices and seeing some strange outbound traffic from them. I realized they were
not playing nice, totally disregarding my configured upstream DNS and instead were using their own company's DNS (8.8.8.8 in Google's case). This is one way these companies manage to transparently track more and more of what you do and have it cost them very little.
Google DNS servers store and track every request made to them and you can bet they are cross referenced back to your device and accounts
<h1>The Solution (mostly)</h1>
There are other ways, but the easiest way is to use a firewall (in my case iptables) I have a Netlink router running the tomato firmware. With this tool we can break down what we want to:
<ul>
<li>All outgoing DNS requests (port 53) from any hosts (other than the pi-hole) should be redirected to the pi-hole.</li>
<li>All outgoing DNS requests (port 53) coming from the pi-hole should be allowed.</li>
</ul>
Assuming your pi-hole's IP address is for example 192.168.1.10, what I ended up with is something like this.
<pre>
iptables -t nat -A POSTROUTING -j MASQUERADE
iptables -t nat -I PREROUTING -i br+ ! -s 192.168.1.10 -p tcp --dport 53 -j DNAT --to 192.168.1.10:53
iptables -t nat -I PREROUTING -i br+ ! -s 192.168.1.10 -p udp --dport 53 -j DNAT --to 192.168.1.10:53
</pre>
<br/>
<br/>
Notes:
<ul>
<li><pre>-i br+</pre>apply to all bridged traffic (this may be different if your router's nic card are named differently?)</li>
<li><pre>-p tcp and -p udp</pre>apply to tcp and udp</li>
<li><pre>! -s 192.168.1.10</pre>apply to all ips except for 192.168.1.10</li>
</ul>
To make this work on Tomato go to Administration > Scripts > Firewall and paste the above 3 lines (with the ip updated) and hit save.
<br/>
<br/>
After doing this, my pi-hole went from blocking 14% of all DNS requests to about 24% (a ludicrous amount in my professional opinion). Success! Even more of my devices stopped firing ads at me at this point. However, there is one final gotcha...
<h1>Unsolved problem 1</h1>
Some companies have figured out that if they don't have their tracking stuff on a distinguishable hostname and instead for example, just in a sub-path that they can get around this specific mechanism of blocking requests.
<h1>Unsolved problem 2</h1>
You will notice also that some browsers like Amazon Silk, which is the only place you can get HD youtube on a fire TV, will still somehow be pushing ads. How is this? Under the guise of an "enhanced experience", Silk renders "content"
(read as: webpages with pre-embedded tracking and ads) on the AWS backend servers and sends it all to you as one big chunk, so your browser never actually makes a DNS request to the tracking site.
<h1>Summary</h1>
In summary, with an afternoon of tinkering and about 50 bucks you too can make your home network safer and more pleasant to use for everyone behind it. For example: My wife noticed immediately that her phone stopped getting ads and was quicker. This
exercise is beneficial but also helped me to really grok just how much unwanted traffic was going on with my network. So get yourself a pi and get on it.
<p><a href="https://benninger.ca/posts/force-dns-go-through-pihole/">How to force all your outgoing DNS queries to go through a pihole</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on July 23, 2019.</p>https://benninger.ca/posts/system76-galago-pro-review-developer2017-08-10T00:00:00-07:002017-08-10T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
I have not actually had a reason to buy a laptop for years, as my old thinkpad serves me just fine for personal stuff and work always provides one.
Recently however I was provided the opportunity to be a guinea pig and select a new Galago Pro while our IT staff was deciding what we should be getting with the caveat that if it doesnt work out and we decide to get something else, I'm stuck with mine. For better or for worse, I said yes. This is a short description of my experience thus far.
I'm not going to get into the unboxing or the "product experience" as youtube is completely awash in those and they add no value to me. However I did notice there are very few reviews after people have used them as a daily driver for a few months and I found that suspicious.
I've now been using my Galago Pro for about 2 months as my main development machine. The specs are as follows:
<ul>
<li>Ubuntu 16.04 (Gnome)</li>
<li>i7-7500U - 2.7GHz-3.5GHz 4MB Cache with 2 Cores and 4 threads</li>
<li>32GB DDR4 @ 2133 MHz</li>
<li>512GB NVMe SSD</li>
<li>867 Mbps Intel Wifi card (the faster one on the customization page)</li>
<li>3200x1800 (16:9) resolution screen</li>
</ul>
<h3>Initial notes and observations</h3>
As with most tech companies, laptops are required to have encrypted disks. The OS that came on it was not setup with disk encryption so I was forced to nuke and pave and reinstall ubuntu 16.04. When I booted up, the wifi did not work. Turns out the intel wifi driver for this card requires kernel 4.8 and higher. Once I installed that kernel things started working. I also added the system76-driver package using their public PPA in order to get their special sauce.
<h3>Pros</h3>
The screen is incredibly vibrant and sharp. The nicest screen I've ever used. (There is a caveat though, see cons)
<br/>
<br/>
The keyboard is fantastic. For a developer this is a biggie and they hit the nail on the head. It feels tight and crisp and the spacing is great. I like the layout as well. Backlight works nicely too and I enjoy it.
<br/>
<br/>
The touchpad is decent. Its not the best but it's probably slightly above par for most consumer laptops.
<br/>
<br/>
It's as fast as one expects given the specs I put in this thing.
<br/>
<br/>
It's very light
<h3>Cons</h3>
The power adapter feels cheap.
<br/>
<br/>
While the screen is incredible, it has now hit home just how badly most linux UI's support HiDPI screens. You can fuss and fiddle with your setup (gnome in my case) and get it to fit beautifully on the small screen but most developers like myself have external monitors and if you have anything less than 3200x1800 on those external monitors, none of the desktops I've tried can handle two radically different scales for each monitor. So to use my two 1080p monitors at work I have to actually disable the laptop screen otherwise it is unusable.
<br/>
<br/>
The battery life is really not good at all. I get around 4 hours, 6 if I'm very conservative and it's likely due to that screen resolution. Before anyone asks, yes I have TLP enabled and powertop has done it's thing on startup. I never thought I'd say this but on laptop this small I regret not getting 1080p.
<br/>
<br/>
The case is only aluminum on the outside edges from what I can tell, where I put my hands it definitely does not feel the coolness of metal. When I first got it, I had somehow missed this fact and was legitimately surprised when I read that it was aluminum.
<br/>
<br/>
Fit and finish is sadly not what I would like to see on something supposedly high end. When you look closely there are tinly little gaps where the case components arent entirely flush with each other and I notice there is a little extra separation at the very top where the webcam is, which gets worse when I open and close the lid causing a bit of flex.
<br/>
<br/>
The fan is more noticible than on any laptop I've ever owned. I understand it's small and needs to use its little fan to get it's job done with minimal space and surface area to make use of, but wow it's startling sometimes. There seems to be a threshold where rather than slightly increasing fan RPM in order to reduce below the threshold, it will just got to 100% RPM for a few seconds and then back down.
<h3>Summary</h3>
All in all, I am enjoying the laptop simply because it's my first small laptop and I like the mobility. I enjoy working on it with the awesome keyboard, the performance and that screen. Because it's for work and just a tool I'll keep it, use it for my work and enjoy doing so. Knowing what I know now, if I was looking for a laptop of my own to purchase with my own hard-earned cash, I might be hesitant to make the purchase for myself.
<h3>Update - Oct 16</h3>
Just a quick update as I've now been using this laptop for over 4 months:
<ul>
<li>The battery life is pretty horrible and 3 hours is really a stretch if you have wifi or anything really going.</li>
<li>The fan is extremely annoying, waking up every 15 or so seconds to go near 80% speed and then turning back off again. It has become an inside joke at work just how noisy it is even under zero load and I no longer bring it to meetings with me as sometimes the fan is audible on conference calls.</li>
<li>The paint in a few small places is getting worn down from contact when the screen is shut. This is a huge bummer as it already is starting to look a bit worn.</li>
</ul>
<h3>Update - Nov 16</h3>
Another quick update
<ul>
<li>I emailed System76 about the fan and to their credit got feedback very quickly. However they said the fan is supposed to sound like that and told me if I press FN+1 I will hear what the fan sounds like at full bore which makes it super loud. In general I would have hoped they might have included a higher-quality fan.</li>
<li>The fan has also started rubbing on something and has now started making a bit of a different noise sometimes. This only confirms that they cheaped out on the fan.</li>
</ul>
<p><a href="https://benninger.ca/posts/system76-galago-pro-review-developer/">A Developer's review of the System76 Galago Pro</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on August 10, 2017.</p>https://benninger.ca/posts/debian-jessie-firefox-html52015-12-08T00:00:00-08:002015-12-08T00:00:00-08:00Chrishttps://benninger.cachris@benninger.ca
I was getting a bit annoyed at all the disjoint posts around the internet about making HTML5 work with my Firefox install on my work laptop so I thought I'd contribute to the noise.
Getting started, if you go to <a href="http://youtube.com/html5">http://youtube.com/html5</a> and you see something like the screenshot below, you might want to try out my instructions.
<br />
<br />
<img src="https://benninger.ca/assets/img/html5-before.png"></img>
<br />
<br />
You might see x264 also in red, this guide should fix that as well.
<h2>Install Debian multimedia repository</h2>
(As root) Add the repo to your sources:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">echo</span> <span class="s2">"http://www.deb-multimedia.org jessie main non-free"</span> <span class="o">>></span> /etc/apt/sources.list.d/deb-multimedia.list</code></pre></figure>
<br />
Update sources and install keyring and update sources again:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>apt-get update
<span class="nb">sudo </span>apt-get <span class="nb">install </span>deb-multimedia-keyring
<span class="nb">sudo </span>apt-get update</code></pre></figure>
<h2>Install Codecs</h2>
Remove any leftover ffmpeg implementation from the original repo:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>apt-get remove ffmpeg</code></pre></figure>
<br />
Install Packages:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>apt-get <span class="nb">install </span>ffmpeg x264 gstreamer0.10-ffmpeg</code></pre></figure>
<h2>Enable Mediasources in Firefox</h3>
Open firefox, go to 'about:config' in your addressbar. Search for 'mediasource'.
<br />
Set the following two keys to 'true':
<pre>media.mediasource.enabled
media.mediasource.webm.enabled</pre>
<br />
Restart Firefox
<h2>Validate Changes</h2>
Go back to <a href="http://youtube.com/html5">http://youtube.com/html5</a> in firefox and hopefully you will see more blue:
<br />
<br />
<img src="https://benninger.ca/assets/img/html5-after.png"></img>
<br />
<br />
Also don't forget to click the "Request the HTML5 player" on youtube to make good use of this.
<h2>Profit!</h2>
See! Linux users can have nice things too.
<p><a href="https://benninger.ca/posts/debian-jessie-firefox-html5/">Making HTML5 work with Firefox on Debian Jessie</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on December 08, 2015.</p>https://benninger.ca/posts/debian-install-california-source2014-12-04T00:00:00-08:002014-12-04T00:00:00-08:00Chrishttps://benninger.cachris@benninger.ca
Decided I wanted to try out the new <a href="https://wiki.gnome.org/Apps/California">California</a> calendar application from the Yorba folks. Sometimes it can be hard to find ever dev library you need on your platform so I thought I'd write up which packages I needed to install.
<br/>First pull down the source from either their git repo or from <a href="https://wiki.gnome.org/Apps/California">here</a>
Then install the following packages on your debian jessie:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>apt-get <span class="nb">install </span>libgtk-3-dev libgee-0.8-dev libecal1.2-dev libsoup2.4-dev libgdata-dev libgoa-1.0-dev libgirepository1.0-dev itstool valac</code></pre></figure>
Now just follow the instructions in the INSTALL file:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">./configure
make</code></pre></figure>
For easy usage you can make a .desktop file with something like this(/home/username/.local/share/applications/california.desktop):
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="o">[</span>Desktop Entry]
<span class="nv">Encoding</span><span class="o">=</span>UTF-8
<span class="nv">Version</span><span class="o">=</span>1.0
<span class="nv">Type</span><span class="o">=</span>Application
<span class="nv">Icon</span><span class="o">=</span>/usr/share/icons/gnome/48x48/mimetypes/x-office-calendar.png
<span class="nv">Exec</span><span class="o">=</span>/path-to-california/california-0.2.0/src/california
<span class="nv">Name</span><span class="o">=</span>California
<span class="nv">Comment</span><span class="o">=</span>California</code></pre></figure>
Now you can just find it in your menu, or in Gnome just hit the 'Super' key and type California.
<p><a href="https://benninger.ca/posts/debian-install-california-source/">Installing the California calendar application on Debian Jessie</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on December 04, 2014.</p>https://benninger.ca/posts/tomcat-authbind-gentoo2014-06-03T00:00:00-07:002014-06-03T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
Today, I tried to move my Tomcat server from 8080/8443 to 80/443. After changing my Tomcat config I restarted the daemon and the log spewed out exceptions about not having permissions to bind to 80 and 443. Example:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">SEVERE: Failed to initialize connector <span class="o">[</span>Connector[HTTP/1.1-80]]
...
Caused by: org.apache.catalina.LifecycleException: Protocol handler initialization failed
...
Caused by: java.net.BindException: Permission denied <null>:80
...
SEVERE: Failed to initialize connector <span class="o">[</span>Connector[HTTP/1.1-443]]
...
Caused by: org.apache.catalina.LifecycleException: Protocol handler initialization failed
...
Caused by: java.net.BindException: Permission denied <null>:443</code></pre></figure>
This is to be expected as ports below 1024 are priviledged ports. There are a <a href="https://answers.atlassian.com/questions/103665/jira-on-port-80-443-using-authbind">number of solutions to this problem</a> but I opted to go with the solution of using <em>authbind</em>.
Following instructions such as <a href="http://www.2ality.com/2010/07/running-tomcat-on-port-80-in-user.html">these</a>, Tomcat still didn't fire up and authbind seemed unable to solve my problem.
I eventually realized it was the way Gentoo handles config files and init scripts and that the common instructions online would not apply.
After some help from one of our local Gentoo guys, we solved the problem in with the following way for Gentoo.
Install authbind
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">emerge authbind</code></pre></figure>
Configure authbind
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">touch</span> /etc/authbind/byport/80
<span class="nb">touch</span> /etc/authbind/byport/443
<span class="nb">chmod </span>500 /etc/authbind/byport/80
<span class="nb">chmod </span>500 /etc/authbind/byport/443
<span class="nb">chown </span>tomcat /etc/authbind/byport/80
<span class="nb">chown </span>tomcat /etc/authbind/byport/443</code></pre></figure>
Authbind doesn't play nice with IPv6, so we need to tell java to prefer it. Edit '/etc/conf.d/tomcat-7-main':
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">...
<span class="nv">JAVA_OPTS</span><span class="o">=</span><span class="s2">"... -Djava.net.preferIPv4Stack=true ..."</span>
....</code></pre></figure>
And finally, tell the Gentoo init script to use authbind and allow child processes to also have the same permission. Edit '/etc/init.d/tomcat-7-main' and change:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">...
start-stop-daemon <span class="nt">--start</span> <span class="se">\</span>
<span class="nt">--quiet</span> <span class="nt">--background</span> <span class="se">\</span>
<span class="nt">--chdir</span> <span class="s2">"</span><span class="k">${</span><span class="nv">CATALINA_TMPDIR</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">--user</span> <span class="k">${</span><span class="nv">CATALINA_USER</span><span class="k">}</span>:<span class="k">${</span><span class="nv">CATALINA_GROUP</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">--make-pidfile</span> <span class="nt">--pidfile</span> <span class="k">${</span><span class="nv">PIDFILE</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">--exec</span> <span class="k">${</span><span class="nv">JAVA_HOME</span><span class="k">}</span>/bin/<span class="k">${</span><span class="nv">cmd</span><span class="k">}</span> <span class="se">\</span>
<span class="k">${</span><span class="nv">JAVA_OPTS</span><span class="k">}</span> <span class="se">\</span>
<span class="k">${</span><span class="nv">args</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">-Dcatalina</span>.base<span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">CATALINA_BASE</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">-Dcatalina</span>.home<span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">CATALINA_HOME</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">-Djava</span>.io.tmpdir<span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">CATALINA_TMPDIR</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">-classpath</span> <span class="s2">"</span><span class="k">${</span><span class="nv">CLASSPATH</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
org.apache.catalina.startup.Bootstrap <span class="se">\</span>
<span class="k">${</span><span class="nv">CATALINA_OPTS</span><span class="k">}</span> <span class="se">\</span>
<span class="k">${</span><span class="nv">TOMCAT_START</span><span class="k">}</span>
...</code></pre></figure>
To this:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">...
start-stop-daemon <span class="nt">--start</span> <span class="se">\</span>
<span class="nt">--quiet</span> <span class="nt">--background</span> <span class="se">\</span>
<span class="nt">--chdir</span> <span class="s2">"</span><span class="k">${</span><span class="nv">CATALINA_TMPDIR</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">--user</span> <span class="k">${</span><span class="nv">CATALINA_USER</span><span class="k">}</span>:<span class="k">${</span><span class="nv">CATALINA_GROUP</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">--make-pidfile</span> <span class="nt">--pidfile</span> <span class="k">${</span><span class="nv">PIDFILE</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">--exec</span> authbind <span class="se">\</span>
<span class="nt">--</span> <span class="se">\</span>
<span class="nt">--deep</span> <span class="k">${</span><span class="nv">JAVA_HOME</span><span class="k">}</span>/bin/<span class="k">${</span><span class="nv">cmd</span><span class="k">}</span> <span class="se">\</span>
<span class="k">${</span><span class="nv">JAVA_OPTS</span><span class="k">}</span> <span class="se">\</span>
<span class="k">${</span><span class="nv">args</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">-Dcatalina</span>.base<span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">CATALINA_BASE</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">-Dcatalina</span>.home<span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">CATALINA_HOME</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">-Djava</span>.io.tmpdir<span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">CATALINA_TMPDIR</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
<span class="nt">-classpath</span> <span class="s2">"</span><span class="k">${</span><span class="nv">CLASSPATH</span><span class="k">}</span><span class="s2">"</span> <span class="se">\</span>
org.apache.catalina.startup.Bootstrap <span class="se">\</span>
<span class="k">${</span><span class="nv">CATALINA_OPTS</span><span class="k">}</span> <span class="se">\</span>
<span class="k">${</span><span class="nv">TOMCAT_START</span><span class="k">}</span>
...</code></pre></figure>
Now Tomcat should fire up:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/etc/init.d/tomcat-7-main stop
/etc/init.d/tomcat-7-main start</code></pre></figure>
<p><a href="https://benninger.ca/posts/tomcat-authbind-gentoo/">Setting up Tomcat on port 80 on Gentoo</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on June 03, 2014.</p>https://benninger.ca/posts/tomcat-vertica-jdbc-gentoo2014-04-21T00:00:00-07:002014-04-21T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
I've been working on some stuff related to <em>Clojure</em>,<em>Scala</em> and <a href="http://vertica.com"><em>HP Vertica</em></a> for work. I ran into the problem where my SBT+Jetty
dev server worked happily, but when packaged as a WAR and deployed to a Tomcat 7 container my webapp wouldnt come up.
When deploying my app, I copied the JDBC driver to <em>$CATALINA_HOME/lib</em> which is the usual location to put JDBC drivers, as this version of Tomcat AFAIK does not read runtime Jars from <em>WEB-INF/lib</em>.
The error message I got was:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">Message: java.lang.NullPointerException: Looking <span class="k">for </span>Connection Identifier ConnectionIdentifier<span class="o">(</span>lift<span class="o">)</span> but failed to find either a JNDI data <span class="nb">source </span>with the name lift or a lift connection manager with the correct name
...</code></pre></figure>
I'll start with a disclaimer that I am not much of a Gentoo guru, but it is what it is. Searching in the catalina logs, I found:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">...
Caused by: java.lang.ClassNotFoundException: com.vertica.jdbc.Driver
...
WARNING: Failed to register <span class="k">in </span>JMX: javax.naming.NamingException: com.vertica.jdbc.Driver
...</code></pre></figure>
After being confused about why Tomcat was not finding my Jar, I read <em>/etc/init.d/tomcat-7-main</em> which lead me to <em>/usr/share/tomcat-7/package.env</em>. I didn't realize but it seems individual Jars are being
specified in this file. I appended my Vertica Jar, and it started working:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">CLASSPATH</span><span class="o">=</span><span class="s2">"...other stuff...:/usr/share/tomcat-7/lib/vertica-jdbc-7.0.1-0.jar"</span></code></pre></figure>
<p><a href="https://benninger.ca/posts/tomcat-vertica-jdbc-gentoo/">Tomcat 7 on Gentoo cannot find Vertica JDBC Driver</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on April 21, 2014.</p>https://benninger.ca/posts/debian-64bit-crashing-google-hangouts-plugin2013-11-12T00:00:00-08:002013-11-12T00:00:00-08:00Chrishttps://benninger.cachris@benninger.ca
After an update the other day, Google Hangouts stopped working in Firefox and in Chrome. The symptoms were a collection of items,
including the browser not registering that the plugin was installed, a prompt saying the plugin has crashed and a message in dmesg
about the plugin segfaulting. It took me a while to find the correct search term and log entry to end at the solution that solved it
for me.
I found the log file in '~/config/google'. After examining I found the following mixed in the messages:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">cat</span> ~/.config/google-googletalkplugin/gtbplugin.log
...
... Connection to GoogleTalkPlugin failed, <span class="nv">reason</span><span class="o">=</span>111
...</code></pre></figure>
After some searching, I ended up finding a <a href="http://productforums.google.com/forum/#!topic/hangouts/vYsaeEnXJXs">thread in Google Support</a> with a number of suggestions.
Ultimately, I found that my update on Debian pulled down 'libudev1' when 'libudev0' was already installed. There was no conflict, but the plugin seemed to use 'libudev0' even though it required 'libudev1'. Simply
<b>uninstalling libudev0</b> was enough to do it for me, but if you need to keep it for some reason, one can force the plugin to use 'libudev1' with a rather-hacky symbolic link:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">ln</span> <span class="nt">-s</span> /lib/x86_64-linux-gnu/libudev.so.1 /opt/google/chrome/lib/libudev.so.0 </code></pre></figure>
I hope this helps someone, as I found it took much longer than I'd like to find this solution, even though it was not that uncommon.
<p><a href="https://benninger.ca/posts/debian-64bit-crashing-google-hangouts-plugin/">Google Hangouts plugin crashes in Debian Testing 64-bit</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on November 12, 2013.</p>https://benninger.ca/posts/bittorrent-sync-on-dns320l-with-funplug2013-09-09T00:00:00-07:002013-09-09T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
This information assumes you have already setup fun_plug on your DNS-320L NAS as per <a href="http://nas-tweaks.net/371/hdd-installation-of-the-fun_plug-0-7-on-nas-devices/"> these instructions</a>. I started by following the following instructions which I found <a href="http://forum.bittorrent.com/topic/19278-sync-to-nas-d-link-dns-320/">here</a>.
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">cd</span> /ffp/start/
wget http://btsync.s3-website-us-east-1.amazonaws.com/btsync_arm.tar.gz
<span class="nb">tar </span>zxvf btsync_arm.tar.gz</code></pre></figure>
Once extracted, I tested it by running:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">chmod </span>x btsync
./btsync</code></pre></figure>
And then accessing *my-nas-ip*:8888 in my browser. The issue I keep seeing people encounter is that they cannot select outside of their '/ffp' folder. This is because the bash shell that funplug starts is run inside a chroot at '/ffp'. However, when you go to select a folder, the root of the filesystem is still in '/mnt'. In my case it is '/mnt/HD/HD_a2/'. It's a simple hurdle, so hopefully this saves a few people a little time.
Once it's working, make it start automatically by adding something like this to your '/mnt/HD/HD_a2/ffp/fun_plug' file:
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">chmod </span>a+x /ffp/start/btsync
/ffp/start/./btsync</code></pre></figure>
Happy syncing!
<p><a href="https://benninger.ca/posts/bittorrent-sync-on-dns320l-with-funplug/">Setting up BitTorrent Sync on a DNS-320L NAS with fun_plug 0.7</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on September 09, 2013.</p>https://benninger.ca/posts/aerofs-problems-on-archlinux2013-09-06T00:00:00-07:002013-09-06T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
AeroFS is a pretty cool (unfortunately not open-source) tool for synchronizing data across your machines. I did bump up against a couple minor issues however which I thought I'd post the solution for. First off if you run Debian, AeroFS provides a <em>.deb</em> which can just be installed, on Arch there is an <a hre="https://aur.archlinux.org/packages/aerofs/">AUR package</a> which, at the time of this writing, is out of date and does not install correctly. So if you arent running a Debian derivative you'll need to install the <em>.tgz</em> version. Other than that there is very little to know.
The main problem I ran into was that when I ran aerofs (or aerofs-cli, from the commandline) it barfed out this useful gem of a message:
<pre>
ERROR: Sorry, AeroFS couldn't launch. Please contact us at support@aerofs.com:
Exit code 127.
</pre>
After some fiddling, I noticed they are using Java under the hood and seeing as how they most likely compiled their code with the standard <em>Oracle Java JDK</em>, my default setup <em>OpenJDK</em> from the Arch repository was probably causing problems (I've had problems before with compatibility with <em>OpenJDK</em>). So I uninstalled the <em>OpenJDK</em> and <em>OpenJRE</em> packages and downloaded the latest <a href="http://java.com/en/download/manual.jsp?locale=en">Oracle JRE</a>. After unpacking it to where I wanted (in this case /opt), I unpacked it and added the following lines to my <em>.bash_profile</em>.
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">export </span><span class="nv">PATH</span><span class="o">=</span><span class="nv">$PATH</span>:/opt/my_jre_version/bin/
<span class="nb">export </span><span class="nv">JAVA_HOME</span><span class="o">=</span>/opt/my_jre_version/</code></pre></figure>
After logging out and back in (or running 'source .bash_profile') AeroFS fires up happily.
<p><a href="https://benninger.ca/posts/aerofs-problems-on-archlinux/">AeroFS on Linux</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on September 06, 2013.</p>https://benninger.ca/posts/how-to-force-snow-leopard-to-consider-a-shell-script-safe2013-06-12T00:00:00-07:002013-06-12T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
You can tell OSX Snow Leopard to consider shell scripts safe to execute without a user prompt. This is useful for automation of certain tasks. You can use any UTI mime-type to allow any filetype you wish. To do so, place the below text in <code>~/Library/Preferences/com.apple.DownloadAssessment.plist</code>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="cp"><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"></span>
<span class="nt"><plist</span> <span class="na">version=</span><span class="s">"1.0"</span><span class="nt">></span>
<span class="nt"><dict></span>
<span class="nt"><key></span>LSRiskCategorySafe<span class="nt"></key></span>
<span class="nt"><dict></span>
<span class="nt"><key></span>LSRiskCategoryContentTypes<span class="nt"></key></span>
<span class="nt"><array></span>
<span class="nt"><string></span>public.shell-script<span class="nt"></string></span>
<span class="nt"></array></span>
<span class="nt"></dict></span>
<span class="nt"></dict></span>
<span class="nt"></plist></span></code></pre></figure>
<p><a href="https://benninger.ca/posts/how-to-force-snow-leopard-to-consider-a-shell-script-safe/">How to Force Snow Leopard+ to consider a shell script safe.</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on June 12, 2013.</p>https://benninger.ca/posts/virtualizing-osx-nic-card-setup2013-05-07T00:00:00-07:002013-05-07T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
If you are automating anything with virtual machines, you likely will be automatically assigning MAC addresses to Nic cards. If you are using any OSX guests, you may have issues around new Nic cards being assigned and having them come up without human interaction. To force OSX to register and bring up new Nics, run the following:
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">networksetup <span class="nt">-detectnewhardware</span></code></pre></figure>
Applying that to the virtualization case, you can use <em>Automator</em> to create and <em>Application</em> that runs an <em>AppleScript</em> with the above command, and set it to run on login.
<p><a href="https://benninger.ca/posts/virtualizing-osx-nic-card-setup/">Virtualizing OSX: Nic card setup</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on May 07, 2013.</p>https://benninger.ca/posts/akkascala-deploying-an-actor-remotely2013-04-26T00:00:00-07:002013-04-26T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
In my use case, I have a variable number of independent <em>ActorSystems</em>. As a new system starts up, it initiates a simple Syn/Ack procedure with the command and control system, allowing the C&C to deploy remote actors. Just for fun, here is some example code.
Lets start with our remote system:
<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="k">object</span> <span class="nc">Remote</span> <span class="k">extends</span> <span class="nc">App</span><span class="o">{</span>
<span class="k">val</span> <span class="nv">system</span> <span class="k">=</span> <span class="nc">ActorSystem</span><span class="o">(</span><span class="s">"RemoteSystem"</span><span class="o">,</span>
<span class="nv">ConfigFactory</span><span class="o">.</span><span class="py">defaultReference</span><span class="o">(</span><span class="nv">getClass</span><span class="o">.</span><span class="py">getClassLoader</span><span class="o">))</span>
<span class="k">val</span> <span class="nv">connStr</span> <span class="k">=</span> <span class="s">"akka://CommandSystem@<ip>:2555/user/CommandActor"</span>
<span class="k">val</span> <span class="nv">cnc</span> <span class="k">=</span> <span class="nv">system</span><span class="o">.</span><span class="py">actorFor</span><span class="o">(</span><span class="n">connStr</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">synActor</span> <span class="k">=</span> <span class="nv">system</span><span class="o">.</span><span class="py">actorOf</span><span class="o">(</span><span class="nc">Props</span><span class="o">[</span><span class="kt">SynActor</span><span class="o">])</span>
<span class="o">}</span>
<span class="k">class</span> <span class="nc">SynActor</span> <span class="k">extends</span> <span class="nc">Actor</span><span class="o">{</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">preStart</span> <span class="k">=</span> <span class="o">{</span>
<span class="c1">//Send the syn
</span>
<span class="nv">Remote</span><span class="o">.</span><span class="py">cnc</span> <span class="o">!</span> <span class="k">new</span> <span class="nc">Syn</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">receive</span> <span class="k">=</span> <span class="o">{</span>
<span class="k">case</span> <span class="n">a</span><span class="k">:</span><span class="kt">Ack</span> <span class="o">=></span> <span class="c1">//
</span>
<span class="o">}</span>
<span class="o">}</span></code></pre></figure>
Some shared classes:
<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="k">case</span> <span class="k">class</span> <span class="nc">Syn</span><span class="o">()</span>
<span class="k">case</span> <span class="k">class</span> <span class="nc">Ack</span><span class="o">()</span></code></pre></figure>
Now the Command and Control system and the actual code you want run remotely:
<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="k">object</span> <span class="nc">Command</span> <span class="k">extends</span> <span class="nc">App</span><span class="o">{</span>
<span class="k">val</span> <span class="nv">system</span> <span class="k">=</span> <span class="nc">ActorSystem</span><span class="o">(</span><span class="s">"CommandSystem"</span><span class="o">,</span>
<span class="nv">ConfigFactory</span><span class="o">.</span><span class="py">defaultReference</span><span class="o">(</span><span class="nv">getClass</span><span class="o">.</span><span class="py">getClassLoader</span><span class="o">))</span>
<span class="k">val</span> <span class="nv">cncActor</span> <span class="k">=</span> <span class="nv">system</span><span class="o">.</span><span class="py">actorOf</span><span class="o">(</span><span class="nc">Props</span><span class="o">[</span><span class="kt">CommandActor</span><span class="o">])</span>
<span class="o">}</span>
<span class="k">class</span> <span class="nc">CommandActor</span> <span class="k">extends</span> <span class="nc">Actor</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">receive</span> <span class="k">=</span> <span class="o">{</span>
<span class="k">case</span> <span class="n">s</span><span class="k">:</span><span class="kt">Syn</span> <span class="o">=></span>
<span class="c1">//Respond for politeness
</span>
<span class="n">sender</span> <span class="o">!</span> <span class="k">new</span> <span class="nc">Ack</span>
<span class="c1">//Get the remote actorsystem path
</span>
<span class="k">val</span> <span class="nv">addr</span> <span class="k">=</span> <span class="nc">AddressFromURIString</span><span class="o">(</span><span class="nv">sender</span><span class="o">.</span><span class="py">path</span><span class="o">.</span><span class="py">root</span><span class="o">.</span><span class="py">toString</span><span class="o">)</span>
<span class="c1">//Push the actor to the remote system
</span>
<span class="k">val</span> <span class="nv">remoteActor</span> <span class="k">=</span> <span class="nv">context</span><span class="o">.</span><span class="py">actorOf</span><span class="o">((</span><span class="nc">Props</span><span class="o">[</span><span class="kt">RemoteActor</span><span class="o">]).</span><span class="py">withDeploy</span><span class="o">(</span>
<span class="nc">Deploy</span><span class="o">(</span><span class="n">scope</span> <span class="k">=</span> <span class="nc">RemoteScope</span><span class="o">(</span><span class="n">addr</span><span class="o">))))</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="k">class</span> <span class="nc">RemoteActor</span> <span class="k">extends</span> <span class="nc">Actor</span> <span class="o">{</span>
<span class="c1">//Where you put your remote code implementation
</span>
<span class="o">}</span></code></pre></figure>
From here, you can handle management of remote actors in whichever way you want. Fun stuff!
<p><a href="https://benninger.ca/posts/akkascala-deploying-an-actor-remotely/">Akka+Scala: Deploying an actor remotely</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on April 26, 2013.</p>https://benninger.ca/posts/creating-an-akka-actor-from-a-dynamically-loaded-jar-file-in-scala2012-11-14T00:00:00-08:002012-11-14T00:00:00-08:00Chrishttps://benninger.cachris@benninger.ca
Creating actors in <a href="http://doc.akka.io/docs/akka/snapshot/scala/actors.html">Akka</a> is generally pretty straight forward. If using the general use-case, there are two slightly different methods to use depending on whether or not your Actor has any parameters in it's constructor.
<!--more-->
<h4>Without constructor parameters</h4>
<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="k">class</span> <span class="nc">MyActor</span> <span class="k">extends</span> <span class="nc">Actor</span> <span class="o">{</span> <span class="o">...</span> <span class="o">}</span>
<span class="k">val</span> <span class="nv">myActor</span> <span class="k">=</span> <span class="nv">system</span><span class="o">.</span><span class="py">actorOf</span><span class="o">(</span><span class="nc">Props</span><span class="o">[</span><span class="kt">MyActor</span><span class="o">],</span><span class="n">name</span><span class="o">=</span><span class="s">"actorname"</span><span class="o">)</span></code></pre></figure>
<h4>With constructor parameters</h4>
<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="k">class</span> <span class="nc">MyActor</span><span class="o">(</span><span class="n">param</span><span class="k">:</span><span class="kt">String</span><span class="o">)</span> <span class="k">extends</span> <span class="nc">Actor</span> <span class="o">{</span> <span class="o">...</span> <span class="o">}</span>
<span class="k">val</span> <span class="nv">myActor</span> <span class="k">=</span> <span class="nv">system</span><span class="o">.</span><span class="py">actorOf</span><span class="o">(</span><span class="nc">Props</span><span class="o">(</span><span class="k">new</span> <span class="nc">MyActor</span><span class="o">(</span><span class="n">parameter</span><span class="o">)),</span><span class="n">name</span><span class="o">=</span><span class="s">"actorname"</span><span class="o">)</span></code></pre></figure>
That's pretty straight forward. However, there are outlier cases for JVM+Scala in which some more dynamic functionality seen in languages like Python is useful. As a result, I've been playing with ways to dynamically load classes from remote jar files. As I have been into Akka+Scala for a while, this ultimately leads to the question: "How can I load an Actor from a jarfile I received from a remote source?". This isn't quite comparable to what Python does, but it is interesting and powerful nonetheless.
Loading a class from a jarfile using a URLClassLoader is fairly straightforward and I wont cover it here. What I will cover is loading an Actor from a URLClassLoader in Scala. Using reflection this can be achieved without too much stress, assuming we do not need to pass parameters into an actor's constructor.
<h4>From a URLClassLoader without constructor parameters</h4>
<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="k">class</span> <span class="nc">MyActor</span> <span class="k">extends</span> <span class="nc">Actor</span> <span class="o">{</span> <span class="o">...</span> <span class="o">}</span>
<span class="k">val</span> <span class="nv">loader</span> <span class="k">=</span> <span class="nf">jarFileToUrlClassLoader</span><span class="o">(</span><span class="n">jarURL</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">clazz</span> <span class="k">=</span> <span class="nv">loader</span><span class="o">.</span><span class="py">loadClass</span><span class="o">(</span><span class="s">"my.pkg.MyActor"</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">myActor</span> <span class="k">=</span> <span class="nv">context</span><span class="o">.</span><span class="py">system</span><span class="o">.</span><span class="py">actorOf</span><span class="o">(</span><span class="nc">Props</span><span class="o">(</span><span class="nv">clazz</span><span class="o">.</span><span class="py">asInstanceOf</span><span class="o">[</span><span class="kt">Class</span><span class="o">[</span><span class="k">_</span> <span class="kt">:</span> <span class="kt"><</span> <span class="kt">akka.actor.Actor</span><span class="o">]])</span></code></pre></figure>
If you're constructor has parameters however, it becomes a little more tricky. Unfortunately the Scala/Akka API doesn't seem to support this. The Java/Akka one does, though we have to use the Creator class. Creator is a factory that makes whatever type you tell it to. Override its create function and return an Actor.
<h4>From a URLClassLoader with constructor parameters</h4>
<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="k">class</span> <span class="nc">MyActor</span><span class="o">(</span><span class="n">p</span><span class="k">:</span><span class="kt">String</span><span class="o">)</span> <span class="k">extends</span> <span class="nc">Actor</span> <span class="o">{</span> <span class="o">...</span> <span class="o">}</span>
<span class="k">val</span> <span class="nv">loader</span> <span class="k">=</span> <span class="nf">jarFileToUrlClassLoader</span><span class="o">(</span><span class="n">jarURL</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">clazz</span> <span class="k">=</span> <span class="nv">loader</span><span class="o">.</span><span class="py">loadClass</span><span class="o">(</span><span class="s">"my.pkg.MyActor"</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">const</span> <span class="k">=</span> <span class="nv">clazz</span><span class="o">.</span><span class="py">getConstructors</span><span class="o">()(</span><span class="mi">0</span><span class="o">)</span> <span class="c1">//Do this properly
</span>
<span class="k">val</span> <span class="nv">prop</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">Props</span>
<span class="k">val</span> <span class="nv">myActor</span> <span class="k">=</span> <span class="nv">prop</span><span class="o">.</span><span class="py">withCreator</span><span class="o">(</span><span class="k">new</span> <span class="nv">akka</span><span class="o">.</span><span class="py">japi</span><span class="o">.</span><span class="py">Creator</span><span class="o">[</span><span class="kt">Actor</span><span class="o">]{</span>
<span class="k">def</span> <span class="nf">create</span><span class="k">:</span><span class="kt">Actor</span> <span class="o">=</span> <span class="o">{</span>
<span class="k">return</span> <span class="nv">const</span><span class="o">.</span><span class="py">newInstance</span><span class="o">(</span><span class="s">"My Parameter"</span><span class="o">).</span><span class="py">asInstanceOf</span><span class="o">[</span><span class="kt">Actor</span><span class="o">]</span>
<span class="o">}</span>
<span class="o">})</span></code></pre></figure>
Tada! Now you have all the tools needed to beam Jarfiles around, load them and fire up actors inside them.
<p><a href="https://benninger.ca/posts/creating-an-akka-actor-from-a-dynamically-loaded-jar-file-in-scala/">Creating an Akka actor from a dynamically loaded Jar file in Scala</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on November 14, 2012.</p>https://benninger.ca/posts/cloud-2012-and-sophoslabs-and-scala-oh-my2012-08-07T00:00:00-07:002012-08-07T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
It's been a long while since I last posted anything, I thought I would summarize a few of the things that have occurred lately. The first of which is that the research paper based on my graduate research was presented at the IEEE Cloud 2012 conference in Hawaii in June. <a href="http://webhome.csc.uvic.ca/~onat/">Dr. Onat Yazir</a> presented the work for me as I wasn't able to attend. For anyone interested, the <a href="http://www.benninger.ca/wp-content/uploads/2012/10/06253540.pdf">paper can be found here</a>.
<br /><br />
In addition to this news, I have accepted a Linux Development position at <a href="http://en.wikipedia.org/wiki/Sophos">Sophos (SophosLabs)</a> in Vancouver BC. I have been there for about a month now and am having a lot of fun. In particular, I've been playing a lot with the <a href="http://www.scala-lang.org/">Scala programming language</a>. In combination with the 3rd-party <a href="http://akka.io/">Akka</a> library, Scala provides an extremely powerful set of tools for building distributed systems using the <a href="http://en.wikipedia.org/wiki/Actor_model">Actor model</a>. I must admit, I originally resisted many of the changes in thinking required for effective use of the Actor model and Scala, especially coming from my favouritism towards C and Python. Scala actors however, have begun to restore my faith in the JVM (what's that you say?). I suggest you give it a shot, I think you'll be pleasantly surprised.
<p><a href="https://benninger.ca/posts/cloud-2012-and-sophoslabs-and-scala-oh-my/">Cloud 2012 and SophosLabs and Scala, Oh my!</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on August 07, 2012.</p>https://benninger.ca/posts/graduation-maitland-and-career-seeking2012-05-29T00:00:00-07:002012-05-29T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
Well it's done, I am finished all the requirements for my MSc. and have now been approved for graduation. \
<br /><br/>
My thesis, entitled <em>Maitland: Analysis of Packed and Encrypted Malware via Paravirtualization Extensions</em> can be found <a href="http://www.benninger.ca/wp-content/uploads/2012/05/Benninger_Christopher_MSc_2012.pdf">here</a> if anyone is interested. In addition, a few months ago we began working on a conference paper version of my thesis work. It was recently accepted to the IEEE Cloud 2012 conference and I am very excited about it. I will post some paper details and a citation once the conference is over (late June).
<br /><br/>
Now that my MSc. is finished, I'll be investigating some career opportunities both here in Vancouver and abroad.
<p><a href="https://benninger.ca/posts/graduation-maitland-and-career-seeking/">Graduation, Maitland and career seeking.</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on May 29, 2012.</p>https://benninger.ca/posts/how-to-get-answers-on-a-developer-or-help-irc-channel2011-11-12T00:00:00-08:002011-11-12T00:00:00-08:00Chrishttps://benninger.cachris@benninger.ca
I've spent the better part of two years working on a very technical project. I don't personally know many people who are able to really help me with the technical side of things. As my environment has little solid documentation, I had nowhere to turn but the developer community's IRC channel and mailing list.
<br /><br/>
When I started out, I was relatively new to the whole "IRC as a method of running a project" thing and found it rather frustrating (sometimes I still do). When I asked for help with a problem I was having, I would usually start out by trying to be civil, introducing myself and framing the problem at a high-level so that people interested or knowledgeable about the area would pipe up. I never got a response and I couldn't understand why.
<br /><br/>
Recently, I'm finding myself on the opposite side of that interaction, ignoring the person asking for help because I don't feel like asking for more information about their issue. For all of the people out there experiencing frustration while trying to get help, I wanted to provide some advice on the subject of getting what you want:
<br /><br/>
<ul>
<li>Culturally, developer communities are usually filled with very technically-mind individuals, each with different skill-sets. They aren't usually interested in using the channel as a form of interaction or a chatroom. They want to solve their problems, keep up to date on the latest developments (by reading other people's posts) and sometimes pipe in when they know something.<br/><br/></li>
<li>Don't bother asking if anyone 'knows about a thing'. Most of these people are at work while on the channel and are working on projects of their own. They aren't watching the channel eagerly to help young bright minds who want to ask high-level questions which demand long, explanatory answers. People are busy! Don't waste their time.<br/><br/></li>
<li>People in these communities are usually most interested in the technical issues. Ask technical questions and provide enough information that they can try and answer it concisely and without asking for more context or information.<br/><br/></li>
</ul>
In summary, when you need a question answered just keep it simple, keep it technical, keep it professional. If you are using software X, instead of asking <em>"Does anyone know anything about X?"</em> try <em>"While using X, I used feature Y and Z happened."</em>, copy error messages, version numbers and everything you can think of that might be useful.
<br /><br/>
Good luck.
<p><a href="https://benninger.ca/posts/how-to-get-answers-on-a-developer-or-help-irc-channel/">How to get answers on a developer or help IRC channel.</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on November 12, 2011.</p>https://benninger.ca/posts/converting-those-pesky-m4a-files-to-mp3-on-linux2011-08-06T00:00:00-07:002011-08-06T00:00:00-07:00Chrishttps://benninger.cachris@benninger.ca
Here is a nice one-liner to convert a .m4a audio file to a .mp3 file:
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="nv">$ </span>faad <span class="nt">-o</span> - input_file.m4a | lame - output_file.mp3</code></pre></figure>
Note: Make sure you have lame and faad installed. In debian they are just 'faad' and 'lame' packages.
<p><a href="https://benninger.ca/posts/converting-those-pesky-m4a-files-to-mp3-on-linux/">Converting those pesky .m4a files to .mp3 on Linux</a> was originally published by Chris at <a href="https://benninger.ca">The Blog of Benninger</a> on August 06, 2011.</p>