If you’re new to data engineering or are a practitioner of a related field, such as data science, or business intelligence, we thought it might be helpful to have a handy list of commonly used terms available for you to get up to speed. This data engineering glossary is by no means exhaustive, but should provide some foundational context and … [Read more...] about Data Engineering Glossary
Data Engineering
Meet Magpie: The End-to-End Data Engineering Platform (VIDEO)
If you’ve been following along with Silectis over the past couple of years, you are familiar with our data engineering platform, Magpie. You’re aware of the many outcomes it puts at the fingertips of data engineers, and teams of data practitioners more largely. If you’re new around here, not to worry. We can catch you up quickly. We are excited to … [Read more...] about Meet Magpie: The End-to-End Data Engineering Platform (VIDEO)
Defining Data Engineering for the Modern Enterprise | Data Week DC Talk (VIDEO)
The practice of data engineering has evolved, and will continue to do so — what does this mean for today’s data engineering teams trying to make sense of it all? So much more than ETL, Silectis believes data engineering within enterprises must adapt to meet the ever-changing needs of the business. In his talk for Data Week DC, CEO Demetrios … [Read more...] about Defining Data Engineering for the Modern Enterprise | Data Week DC Talk (VIDEO)
The Case for the Holistic Data Engineering Platform
We recently published a new report: The Case for the Holistic Data Engineering Platform, which outlines why piecemeal tooling isn't sustainable for most modern companies. In today’s world, it’s well understood that the best businesses make data-driven decisions to push their organizations forward. The business landscape is largely a race - to be … [Read more...] about The Case for the Holistic Data Engineering Platform
Apache Spark: A Beginner’s Guide to Optimizing Spark Scripts
This blog post covers Apache Spark basics and teaches readers why optimizing Spark scripts is important, and how to do it for both memory and runtime efficiency. This blog post is best suited for data analysts and data scientists looking for information on optimizing existing Spark workflows or creating new ones. INTRODUCTION: WHAT IS APACHE … [Read more...] about Apache Spark: A Beginner’s Guide to Optimizing Spark Scripts
Setting Up Your Data Engineering Team for Success: Building Future-Proof Data Architecture
Yesterday's data is predictable and easy. Our challenge as data engineers is getting ready for tomorrow's unpredictable new data. With the push toward more flexible solutions, let's discuss how you can set up your data engineering team for success by implementing flexible, scalable, and future-proof data architecture. DATA IS BIG AND GROWING IN … [Read more...] about Setting Up Your Data Engineering Team for Success: Building Future-Proof Data Architecture