Data Engineering
Apache Kafka provides distributed log store used by increasing numbers of companies and often forming the heart of systems processing huge amounts of data. This post shows how to use it for storing meteorological data and displaying this in a graphical dashboard with Graphite and Grafana
Tech
JUnit 5 has been available for a few weeks now so in this post I will investigate setting it up and migrating from JUnit 4 then explore some of the new features available.
Delivery
Working for a Digital agile consultancy was my first real exposure to Government / Public Service projects. GDS provide an exceptional amount of process and control for any OGD (Originating Government Department) so trying to follow their rules, the client's wishes and employing a succesful agile delivery methodology can mean serving multiple masters. It did (sometimes) work out.
Data Engineering
A discussion about Cassandra consistency levels and replication factor, which are frequently misunderstood. This post explains the Cassandra infrastructure and how its configuration can be tuned.
Testing
Having good tests is vital for maintaing code but it is difficult to assess the quality of your tests. Mutation testing provides one way of evaluating your tests. In this post I will be using PIT with Java to demonstrate the capabilities of mutation testing.
Delivery
If you're new to Agile you've probably heard the term Story Point a few times and, especially if you come from a more traditional project management or product background, you may be trying to figure out what a Story Point is and how big one is. This article tries to explain what this is in a more digestible format.
Tech
Creating asynchronous functions that are recursive can be a bit of a challenge. This blog post takes a look at various different approaches, including callbacks and promises, and ultimately demonstrates how async functions result in a much simpler and cleaner solution.
People
As the Edinburgh 2017 Summer Interns, Arran Rashid and I have spent twelve weeks working on a new foyer screen for the Scott Logic offices, this post describes the project and our time at Scott Logic.
Delivery
With a well publicised history of multi-billion pound IT failures and operating inefficiencies the UK government took the decision to adopt agile across the public sector. This is a brief look at the history of that adoption and lessons learned for those with an interest in agile project delivery.
Tech
Jest is a testing framework that provides the testing tools we now expect to see in a modern software project. It provides fast parallelised test running, with a familiar assertion syntax, built in code coverage, Snapshots and more. In this post, I'll be investigating Snapshots and laying out some thoughts!
Data Engineering
Lichess makes over 100GB of chess games from 2017 available on their website. This post shows how this data can be transformed with Apache Spark and analysed. Something for Data Engineers and Chess Enthusiasts alike!
People
This blog post, written by a graduate developer joining the Edinburgh team, reflects on his induction week experience at Newcastle. A total of sixteen graduates across UX, development, and testing joined this August, and the induction was an ideal opportunity for everyone to meet.
Tech
This post demonstrates one way to add Selenium e2e tests to a `create-react-app` generated app. It uses Jest as the test runner, embraces modern JavaScript features (including async/await, modules, fat-arrow functions, templated strings, etc.) and attempts to ensure that modern editors are able to provide useful auto-complete suggestions when authoring tests.
Tech
Market profile charts are relatively complicated and specialised charts used by financial traders. This blog post looks at how to render a market profile chart using D3.
Delivery
This blog post discusses concerns around agile delivery, particularly within the public sector
Data Engineering
Yesterday the Financial Times boldly declared that BP saved $7bn since 2014 by investing in Big Data technologies. I spent a couple of hours researching Big Data technologies associated with BP members of staff to try and build up a picture of exactly which technologies they are using.
Data Engineering
Using microservices in your architecture is a very popular choice. Unfortunately it is also challenging to get it right. With the help of Twelve-Factor methodology, I will tell you how to set yourself up for a success rather than a disappointment.
Open Source
A few months ago we contributed a project, ContainerJS, to the Symphony Software Foundation, an organization that fosters open source and collaboration within financial services.
UX Design
In this post, I go over the similarities and differences between Sketch and Illustrator when working with type.