Category Archives: Engineering

Rustconf 2016 – What Was Cool And What Surprised Me

Rustconf 2016 – What Was Cool And What Surprised Me At AgilData, we’re building an upcoming product in Rust so we are pretty heavily invested, which is why I was more than excited to attend the first ever annual Rust conference this past weekend (9/9-9/10). If you don’t know about Rust already, it is a…

Read More

Apache Spark 2.0 API Improvements: RDD, DataFrame, Dataset and SQL

Following on from our previous blog post, Apache Spark: RDD, DataFrame or Dataset?, here is an updated guide to the main Scala and Java APIs for the recently released Spark 2.0 Apache Spark 2.0 API Improvements: RDD, DataFrame, Dataset and SQL What’s New, What’s Changed and How to get Started. Are you ready for Apache Spark…

Read More

AgilData CEO Dan Lynn Talks About Analytics and Apache Kudu at the United Nations, Database Camp 2016

On July 10, 2016, the United Nations hosted one of the largest open source technology events in the world – Database Camp. Open Camps @ UN 2016 brought together developers from open source communities, private sector technology companies, academic institutions and Member States to collaborate on open source technology solutions that support the Organization’s mission….

Read More

10 Reasons We Like Kudu as Part of Your Big Data Strategy

Why Kudu Should be Part of Your Big Data Strategy AgilData is an early adopter, implementer and operator of Kudu using it for one of the first production sites at a leading Insurance Analytics company.  We are contributing code to the project, as we go, having added a considerable performance improvement to the Java driver….

Read More

Data Pipelines with Alooma for BI Analytics

Data Pipelines with Alooma for BI Analytics. Making it Simple In the world of Big Data, the increasingly common task of pipelining data from disparate data sources into an OLAP friendly medium can quickly get complex and messy, especially when there are multiple data sources. This article is for the teams who are not already…

Read More

Mesos Docker Tutorial: How to Build Your Own Framework

Mesos Docker Tutorial: How to Build Your Own Framework Introduction Apache Mesos is a cluster manager that simplifies the complexity of running tasks on a shared pool of servers. Docker is a lightweight container for deploying packaged services, similar in concept to a virtual machine, but without the overhead. Mesos added support for Docker in…

Read More

Packet Capturing MySQL with Rust

Packet Capturing MySQL with Rust Recently, AgilData launched the Gibbs MySQL Scalability Advisor, a free self-service tool that allows users to capture a live stream of queries to be uploaded to Gibbs and analyzed by AgilData’s experts.  Spyglass is the database traffic capture tool for Gibbs. Built using the Rust programming language, it provides exceptional…

Read More

Apache Spark Cluster Managers: YARN, Mesos, or Standalone?

Which Apache Spark Cluster Managers Are The Right Fit? YARN, Mesos, or Standalone? Trying to decide which Apache Spark cluster managers are the right fit for your specific use case when deploying a Hadoop Spark Cluster on EC2 can be challenging. This post breaks down the general features of each solution and details the scheduling,…

Read More

SQL on Hadoop The Differences and Making the Right Choice

SQL on Hadoop : The Differences and Making  the Right Choice Whether you are on Hadoop now or considering making the leap, this article will help you understand the various SQL on Hadoop tools, how they compare, and how they stack up against each other.  The powerful (and ever growing) ecosystem of Hadoop is, and…

Read More

Apache Spark: RDD, DataFrame or Dataset?

There Are Now 3 Apache Spark APIs. Here’s How to Choose the Right One See Apache Spark 2.0 API Improvements: RDD, DataFrame, DataSet and SQL here. Apache Spark is evolving at a rapid pace, including changes and additions to core APIs. One of the most disruptive areas of change is around the representation of data…

Read More

Welcome to the AgilData Blog!

Find the latest news about AgilData, Big Data and tips and tricks straight from our engineering team.

Top