The "gaps and islands" problem is a scenario in which you need to identify groups of continuous data (“islands”) and groups where the data is missing (“gaps”) across a particular sequence.The problem may sound arcane at first glance but knowing how to solve this type of problem is critical for advanced SQL practitioners. These starting and … [Read more...] about Improve Your SQL Skills: Master the Gaps and Islands Problem
MAGPIE IS THE EASIEST WAY TO SETUP YOUR DATA LAKE ON GOOGLE CLOUD PLATFORMToday, Silectis is excited to announce our data engineering platform, Magpie, supports Google Cloud. Silectis Magpie brings together Google Cloud’s Big Data components so that organizations can get value from their data engineering operations right away, providing a … [Read more...] about Magpie Supports Google Cloud Platform
Let’s face it, operating in a data-driven environment is hard. Teams, even small ones, can generate a painfully large number of batch processes that need to run on schedules. Drag-and-drop ETL tools become a maze of dependencies as business logic expands. Cron jobs lack transparency, failing silently and sucking away developer time. It’s in … [Read more...] about How to Implement a Data Lake with Apache Airflow and Silectis Magpie
Today’s advanced analytics space is filled with tools to help users become more data driven. However, every use case is different, and there is not a single tool that is ready for each unique situation. The best solutions for a given organization usually lie in a combinations of software platforms. This blog post outlines why Silectis Magpie and … [Read more...] about Magpie and Looker Partner to Supercharge Your Data Analytics & Visualizations
This blog post was written in conjunction with UrbanTurf, the authority on real estate in the Nation’s Capital, and can be found here.SOLAR POWER IS COMING TO DC… BUT NOT WHERE YOU WOULD THINKSilectis used public building permit data from DC’s open data portal to determine which neighborhoods in Washington DC really care about the future of … [Read more...] about Magpie in Action: Analyzing the Future of Solar Power in Washington, DC
In this blog post, we explore how a data analyst can know when to trust data and when to be skeptical. Using data integrity features in your modern data engineering platform should help you sort this quickly, and move on to empowering decision-making across the company.DATA INTEGRITY: JUST THE FACTSPeople say you can trust data. “Let … [Read more...] about Data Integrity Issues Holding You Back?