We recently published a new report: The Case for the Holistic Data Engineering Platform, which outlines why piecemeal tooling isn't sustainable for most modern companies. In today’s world, it’s well understood that the best businesses make data-driven decisions to push their organizations forward. The business landscape is largely a race - to be … [Read more...]
How CARFAX Uses Magpie to Modernize Its Data Architecture
At CARFAX, a data services company that supplies vehicle history reports to individuals and businesses, accuracy and trustworthiness in their data is paramount, and a modern data architecture is their vehicle for sound decision-making across the company. Karen White, Director of Business Intelligence, recently led her team’s transformation from … [Read more...]
Tutorial: Creating a Multi-Cloud VPN with Terraform between AWS, GCP, and Azure
At Silectis, we deploy Magpie clusters across AWS, Google Cloud, and Azure. But because some of our internal infrastructure resides only on AWS, we need to establish private connections between these environments so that clusters on Google Cloud and Azure can access those private AWS resources. In this post, we’ll walk through the details of how we … [Read more...]
Data Lake Architecture Guide: Choosing the Right Storage Tool
Overview: Build Your Data Architecture to Enable Use Cases One of the things that we often wrestle with in building out data lake architecture is how to best lay out the infrastructure to support different analytical use cases, and more specifically, what storage mechanism might yield the best performance. One of the virtues of data lakes is … [Read more...]
Want to be Data-Driven? Start with Metadata
This blog post gives a brief overview of the importance of metadata in today’s world… with a nod to our school days. HOW THE DEWEY DECIMAL SYSTEM IS KIND OF LIKE METADATA The Dewey Decimal System is an often forgotten example of the value of metadata. Though increasingly arcane, the system gave a library visitor information about a … [Read more...]