At Silectis, we deploy Magpie clusters across AWS, Google Cloud, and Azure. But because some of our internal infrastructure resides only on AWS, we need to establish private connections between these environments so that clusters on Google Cloud and Azure can access those private AWS resources. In this post, we’ll walk through the details of how we … [Read more...] about Tutorial: Creating a Multi-Cloud VPN with Terraform between AWS, GCP, and Azure
Magpie
Magpie Supports Google Cloud Platform
MAGPIE IS THE EASIEST WAY TO SETUP YOUR DATA LAKE ON GOOGLE CLOUD PLATFORM Today, Silectis is excited to announce our data engineering platform, Magpie, supports Google Cloud. Silectis Magpie brings together Google Cloud’s Big Data components so that organizations can get value from their data engineering operations right away, providing a … [Read more...] about Magpie Supports Google Cloud Platform
How to Implement a Data Lake with Apache Airflow and Silectis Magpie
Let’s face it, operating in a data-driven environment is hard. Teams, even small ones, can generate a painfully large number of batch processes that need to run on schedules. Drag-and-drop ETL tools become a maze of dependencies as business logic expands. Cron jobs lack transparency, failing silently and sucking away developer time. It’s in … [Read more...] about How to Implement a Data Lake with Apache Airflow and Silectis Magpie
Magpie and Looker Partner to Supercharge Your Data Analytics & Visualizations
Today’s advanced analytics space is filled with tools to help users become more data driven. However, every use case is different, and there is not a single tool that is ready for each unique situation. The best solutions for a given organization usually lie in a combinations of software platforms. This blog post outlines why Silectis Magpie and … [Read more...] about Magpie and Looker Partner to Supercharge Your Data Analytics & Visualizations
How to Ensure Security In Your Data Lake
THE IMPORTANCE OF SECURITY This post is the third in a series of posts about getting up and running with a Magpie Data Lake. In previous blog posts, we’ve discussed rapidly prototyping a data lake with Magpie, and automating loads into a data lake with Magpie. This post will address a third important piece of data lake infrastructure: security … [Read more...] about How to Ensure Security In Your Data Lake
Tutorial: Using Magpie to Implement a Cloud Data Lake
PILOTING A DATA LAKE WITH MAGPIE More and more companies are turning to data lakes as a way to unify and get value out of their growing collections of data. However, it can be a challenging to navigate the ever-changing technology landscape around these lakes, set one up, and quickly get value from it. At Silectis, we recommend that our … [Read more...] about Tutorial: Using Magpie to Implement a Cloud Data Lake