Advertising and Malware block list
Posted: 2018-09-28 08:59:15 by Alasdair Keyes
Via a Hacker News article https://news.ycombinator.com/item?id=18075159 I came across a link to this git repo https://github.com/StevenBlack/hosts .
It's a curated list of known advertising and malware hostnames that can be added to your system hosts file to protect your system from shady sites accessed by your browser as part of included javascript/tracking pixels etc. As long as your browser still uses your system resolver instead of being set to DNS over HTTPS it should provide a good degree of security for your browsing.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Mojolicious v8 release
Posted: 2018-09-19 11:14:22 by Alasdair Keyes
Perl 5 is often seen as a dated, if not dead language, consigned to being hacked together by a sysad to keep some legacy platform from the 90s ticking over.
Sadly that part is true, but there is still life in the old dog yet, helped by the recent release of version 8 of the Mojolicious framework.
If you've not played with Perl but you have an interest, Mojolicious is a good entry point and I'd recommend it for any new projects. It's built in support for web-sockets makes it very suitable for modern back-end platforms.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Github and Gitlab user public keys
Posted: 2018-09-02 11:18:15 by Alasdair Keyes
During installation of Ubuntu Server 18.04 today I noticed that the user creation section had an option for configuring SSH public keys from Github, this is a very useful feature but I was intrigued as to how these were exposed. It turns out that you can use the following URLs to get the keys for a given user for Github and Gitlab
https://gitlab.com/<username>.keys and https://github.com/<username>.keys
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Nagios NRPE wireguard check
Posted: 2018-08-30 16:41:36 by Alasdair Keyes
After playing around with Wireguard today (of which I'll no doubt do a more fully featured post). I wrote a quite Nagios NRPE check. There's not much to check with Wireguard, but it queries the wg show output for the number of interfaces and peers. Full arguments are in the README. Let me know if you have any suggestions for future behaviour.
https://gitlab.com/alasdairkeyes/nagios-plugin-check_wireguard
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
akeyes.co.uk - Now on HTTP/2
Posted: 2018-08-29 14:48:04 by Alasdair Keyes
Since HTTP/2 was implemented in NGINX, I've wanted to use it on my sites. Unfortunately due to one of my VPS providers not providing a Debian 9 image, I was stuck on Debian 8 with an NGINX version that didn't support it.
Now that I've upgraded all my machines to Debian 9 I have finally updated my sites to make use of HTTP2. Combined with the upgrade to PHP7.x at the same time it should allow some performance increase for my sites.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
LXC config error after update
Posted: 2018-07-19 21:46:48 by Alasdair Keyes
At some point in the recent past Linux Mint 18.3/Ubuntu 16.04 Xenial had updated lxc containers to a more recent version and when running lxc-ls I was greeted with the following error for all of my containers.
# lxc-ls
Failed to load config for mydevcontainer1
Failed to load config for mydevcontainer2
Failed to load config for mydevcontainer3
Failed to load config for mydevcontainer4
It appears support for an older container config version has been discontinued and it was unable to read the old versions, a bit of Googling found the answer was to use lxc-update-config helper utility.
# lxc-update-config -c /var/lib/lxc/mydevcontainer/config
# lxc-start -n mydevcontainer
#
For those with a lot of containers, the following one liner will resolve them all for you in one fell swoop
lxc-ls 2>&1 | grep "Failed to load config for" | awk '{print $NF}' | xargs -i lxc-update-config -c /var/lib/lxc/'{}'/config
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
From Github to Gitlab
Posted: 2018-07-08 16:54:20 by Alasdair Keyes
I've had a Gitlab account for a couple of years because of their facility for free Private repos and since then I have pondered moving all my Public Github repos across but never had the inspiration. The main reason not to was because Github was the de-facto Source Control site, so when dealing with non-tech people like recruiters, it was easier to just point people at Github rather than explaining the lesser known (at the time) Gitlab.
Now with the acquisition of Github by Microsoft, I've got the inspiration. With the exodus of many repos to Gitlab, they now have a much greater reputation outside the tech-circle so my code has now been moved.
The Github projects still exist but just have a README.md directing people to Gitlab.
https://gitlab.com/alasdairkeyes
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Perl Capture::Tiny
Posted: 2018-05-31 08:01:22 by Alasdair Keyes
For such a versatile language, Perl has no good built-in way of running an external commands.
You can obviously use exec(), system() or even the dreaded back ticks ``. But often when running commands you require different things. Maybe you just need an exit code, so system() is good. If you're looking to capture output, exec() or backticks are more suited for the job, but then escaping input and using pipes or redirects then becomes a nightmare.
Last year worked on a project that required a large amount of systems integration and calling binaries, to run tasks and process output so I had to find a reliable and useful method. Mixing system(), exec() was possible but as I would be handing the code off to the customer, having a single way of running commands is a lot cleaner.
I came across https://metacpan.org/pod/Capture::Tiny. It provides a clean interface for capturing output from both Perl code and running system binaries.
#!/usr/bin/env perl
use strict;
use warnings;
use Capture::Tiny ':all';
use v5.20;
my ($stdout, $stderr, $exit_code) = capture {
system("ls", "/var");
};
say "STDOUT\n$stdout";
say "STDERR\n$stderr";
say "EXITCODE: $exit_code";
STDOUT
backups
cache
games
lib
local
lock
log
mail
opt
run
spool
tmp
STDERR
EXITCODE: 0
The only flaw seems to be that it doesn't handle the capturing of output long running processes that will spit out text slowly. For that you will want to use something like pump on https://metacpan.org/pod/IPC::Run
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
Quick wins for your dev team
Posted: 2018-05-30 17:07:47 by Alasdair Keyes
The joy of contracting is that I often get to work on brand new code-bases with some regularity. The down side is exactly the same.
Over a number of contracts I've noticed a number of things that I've suggested to teams that make their development much easier.
If you work for a fair-sized business or a 'tech' company you can stop reading here, you will look at these suggestions, scoff, and wonder why they need making. However there are a lot of companies that are not 'tech' focused. Their website functionality was bolted onto an existing business that pre-dates the web and the first site was a few scripts written by the most tech-savvy employee they had at the time and grew organically from there. Yes, these code-bases are often a mess and constant tight deadlines for new features and product means it always stays that way and doesn't get improved, but that's reason not to try. It's usually the devs that have to work with the code that get most jaded with the situation, having to work with the code day-in and day-out.
These suggestions are mainly based around PHP, but they can be easily adapted for any other language.
Version Control
This might seem like a no-brainer, but I still see teams without Version control (or still stuck with CVS/SVN). Creating an account through Gitlab is highly recommended, not only do you get their git system, they also have free CI/CD pipelines to allow automated testing which will come in handy if you wish to use these later.
Upside
- Well, you get version control. Multiple devs can work on the same code with less risk of implementing breaking changes.
Downside
- If your team are unfamiliar, a little training may be required to get going. A morning reading up on Gitflow and a few articles on merging will be more than enough to familiarise a team with basic concepts and branching/merging.
Consistent Clean Code Formatting and naming
We've all seen code like this, or if you're unlucky with even less line breaks or line breaks in crazy places
public function fib($n) {if ($n < 0) { return NULL; }
elseif ($n === 0) { return 0; } elseif ($n === 1 || $n === 2) { return 1; }
else { return fibonacci($n-1) + fibonacci($n-2); }}
The code's purpose is not easily understood and the name fib doesn't instantly describe what the function is doing.
PSR-2 is the PHP Framework Interoprability Group's approach which has become the defacto standard for PHP and will make your code much nicer to work with.
Along with nice naming you can turn the above into...
public function fibonacci($n) {
if ($n < 0) {
return NULL;
} elseif ($n === 0) {
return 0;
} elseif ($n === 1 || $n === 2) {
return 1;
} else {
return fibonacci($n-1) + fibonacci($n-2);
}
}
You don't have to strictly adhere to a well-known standard, but using a consistent, clean formatting approach can make your life just a little bit easier.
You can look at doing it in one fell swoop with a tool such as PHP Coding Standards Fixer or do a gradual merge whereby all files that are edited with new code get reformatted before any changes and committed to the repo. Tools such as PHPStorm also have a formatting feature available using CTRL+ALT+L
Upside
- You'll restore some sanity to your devs
Downside
- Reformatting large swathes of your code in one fell swoop will make you nervous. A more cautious approach could be to reformat code before you do new work on a class/file, reformat it, commit it, then do your work.
Adopt a standard documentation format
Each language has it's own documentation format PHPDoc for PHP, Perldoc for Perl etc. What you use isn't so much the key as just using it. A quick one line introduction to a class/function plus documentation on the arguments expected and return values will make your code much more pleasant for people unfamiliar with it.
Again, a rolling adoption is easy and fairly hassle free. Write the required docs for new functions, and then as you edit existing ones, back-fill your documentation. In addition if you use code reviews, get people to check documentation updates as well as the code being committed.
Upside
- Over a short amount of time, you can generate usable documentation for your code base.
- If your IDE supports parsing of documentation like PHPDoc/PHPStorm, then it will speed up your development by showing you function arguments and alerting on exceptions that need to be caught without having to search through your code.
Downside
- There isn't really any, it only adds a few minutes more time spent developing your code.
Basic tests
The chances are if you have not been writing tests from the start, it will not be easy to start introducing tests to old code. The primary reason is probably you won't have written your code in a manner to allow easy testing, you won't be injecting dependencies and your functions will be performing too much work to create concise tests.
Don't let this put you off. Simple tests can be put in place quite easily for even the most attrocious code base.
Firstly, if your team is unfamiliar, take a morning to read up on the different types of tests commonly used (unit, integration, functional, Behaviour etc), the benefits they provide and when you are likely to use them. There are plenty of pages and Youtube videos available via Google to get you started.
Secondly, look through your code base for simple functions that are able to have unit tests created for them. These might be few and far between to start with, but once the tests are written, you can get a little more piece of mind that some future commit is less likely to break functionality without anyone noticing.
If you use a common framework, most of these provide a customised test suite to enable rapid test development for WebApp functionality. A simple, quick and useful test could be to ensure that all access to login restricted URLs returns a suitable message and/or HTTP status code along with a redirect to a login page. This can then perhaps be extended to test valid/invalid login messages and redirects. It's a small step, but you're heading in the right direction.
Testing is a huge subject and you could easily get drawn down a rabbit hole, however, with just a few test, you are already more certain of stopping some errors creeping into your code.
Upside
- Basic functionality can be tested with little overhead to ensure that your code is doing what it should be.
Downside
- Getting more than some basic tests written will likely require a lot of refactoring of your code... which requires tests to ensure you don't break anything, it can get messy if not tackled correctly.
This isn't an exhaustive list, if you find that you are in need of implementing some of these you will likely find that there are lots more changes required in your development processes, but these will allow you to gain a lot of benefit without tying up devs for days or months.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
expandtab in vim 8 on Debian 9 not working
Posted: 2018-05-28 15:41:44 by Alasdair Keyes
Last week I finished rolling out my new Debian 9 "Stretch" machines and noticed some peculiar behaviour in vim. Namely, my tabs were being kept as tabs and not converted automatically to spaces. We'll ignore the old "tabs vs. spaces" debate for the moment but I was struggling to see why my tried and tested vim config wasn't working.
My .vimrc file had the following lines
" Set tabs to 4 spaces
set shiftwidth=4
set ts=4
set expandtab
" Allow simple pasting and add ruler/linecounts
set paste
After some hunting I found the following article explaining the issue https://bugs.launchpad.net/ubuntu/+source/vim/+bug/1576583
Debian 9 moves to vim version 8.0 and it seems that the behaviour of paste has changed so that it resets expandtab. All that's required is to move the set paste line before expandtab and you're back in business. Hopefully this might help someone else who has seen the same issues.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
© Alasdair Keyes
IT Consultancy Services
I'm now available for IT consultancy and software development services - Cloudee LTD.
Happy user of Digital Ocean (Affiliate link)
Version:master-28fc6e6b4b
