Andrew Carr
Blog

Andrew Carr

I lead the Data Engineering Practice within Scott Logic. I have a strong interest and expertise in low latency Front Office trading systems, software managing very large networks and the technologies involved in processing large volumes of data.

Data Engineering
What started as one faulty gas reading in the summer of 2017, ended up as a series of wasted calls where my bill kept getting higher and higher until it reached £11k. How could this have been handled faster and left me without considering moving energy provider.
Data Engineering
Distributed stream processing engines have been on the rise in the last few years, first Hadoop became popular as a batch processing engine, then focus shifted towards stream processing engines. Stream processing engines can make the job of processing data that comes in via a stream easier than ever before.
Data Engineering
Distributed log technologies have matured in the last few years. In this article, I review the attributes of distributed log technologies, compare four of the most popular and suggest how to choose the one that's right for you.
Resources
Data lakes? Most Financial Service organisations either have one or are considering one. This white paper explains key considerations and a warning to help guide their creation or optimisation to fulfil its requirement
Data Engineering
Yesterday the Financial Times boldly declared that BP saved $7bn since 2014 by investing in Big Data technologies. I spent a couple of hours researching Big Data technologies associated with BP members of staff to try and build up a picture of exactly which technologies they are using.
Data Engineering
Apache Spark has quickly become the largest open source project in big data, but why has it suddenly got so much momentum behind it?
Data Engineering
Welcome to my blog, Andrew's thoughts on Big Data. This page gives a little background on myself and the blog. My name is Andrew Carr and I have been coding and generally being interested in technology since the age of 12. I have been involved in professional Software Engineering for...