Start here
What does it mean to be a modern data engineer?
If you are working as a data engineer or aiming to become one, you face more complexity than ever.
You’re expected to:
Keep systems and pipelines stable and running
Support new data initiatives with data that drives business decisions
Ensure data quality your stakeholders can trust
Set up observability and monitoring so issues are detected early by your team and not brought up in the next stakeholder meeting
Establish standards for security, data management, and governance for the company according to GDPR and CCPA regulations
Build systems that are scalable, maintainable, and reliable
Stay current in a fast-changing tool landscape, so your career won’t fall flat
Help your team level up through ownership and collaboration
If you are like me, you probably agree that balancing these task sounds almost impossible.
Yet, solving that question is exactly what Modern Data Engineering is all about.
It’s about developing a strategy and the skills for building better systems, becoming a better engineer and having more impact (without losing your mind in the process).
Welcome to Modern Data Engineering
Being a data engineer today means working in a world of LLMs, automations, and tools with ever increasing capabilities. There’s less grunt work and more opportunity to elevate our role.
Yes, we still build pipelines, do coding, and clean data. But modern data engineering goes further.
It’s about becoming:
The Expert Builder crafting production-grade systems with confidence.
The Strategic Engineer applying architectural thinking to modeling, scalability, governance, and design trade-offs.
The Force Multiplier lifting your entire team through ownership, clarity, and leadership.
These are the three pillars of modern data engineering and this publication helps you master all of them.
The three pillars of modern data engineering
The three pillars are about becoming the kind of engineer who…
Builds robust systems
Knows what to build and why, and what happens over time
Elevates the team, the product, the company
Let’s look at them briefly.
The Expert Builder
Even today, data quality and robust systems are often overlooked. But, if you want to avoid being stuck in maintenance mode most of your workday, you need to become a master of your craft.
Obviously, this includes knowledge of the tools. But most data engineers stop here and forget the parts that truly matter.
I’m a big believer in mastering the fundamentals first, before becoming an expert in tool X or platform Y.
“Get the fundamentals down and the level of everything you do will rise.”
Michael Jordan
Tools come and go. If you want to stay sharp and relevant, focus on the parts of your job that will still matter years from now.
Here are some of the technical fundamentals that separate good builders from great ones:
SQL & Python
Naming conventions, configs, structure, versioning
CI/CD principles
Testing your code and data
Automation and observability at the pipeline level
… and more.
These aren’t just “best practices”. They’re what protect your team from fires, your stakeholders from errors, and your reputation from silently breaking.
In The Expert Builder section, you won’t just find how to do things - but how to do things right. We’ll strive not just for shallow knowledge, but for deep understanding them.
One post at a time.
The Strategic Engineer
The best data engineers don’t just build what is described in tickets.
They think like architects - designing for performance, maintainability, and change.
They understand why certain tools or patterns work, anticipate bottlenecks, advocate for governance, quality, and long-term clarity.
In short: They own the process.
Some of the core topics include:
Data modeling, architecture, lineage, contracts
Scalability, idempotency, reproducibility
Data governance and data management
DataOps, monitoring, and testing strategies
Cost optimization, schema evolution, design trade-offs
This is where you grow from technician to engineer.
Not just “how do I build this?” but what to build, why it matters, what could go wrong, and how it should evolve.
The Force Multiplier
There was a turning point in my career.
After several low impact roles and eventually being seen as the “lead tech guy”, I got into a new project.
The situation was rough. I was working for an agency whose client just scaled the user base from a few thousands to millions of active users. The product was growing, the client was happy.
The problem? The entire team (eight people) had left over the previous 12 months. That left me as the most experienced team member with less than two months of project experience.
The client trusted the product, but they no longer trusted the process.
They were nervous and I could sense it from day one.
I realized something quickly:
Doing a technically solid job over the next few months wouldn’t be enough.
I had to evolve and be more than the leading tech guy. I had to:
Ensure every release was spot-on with no critical issues
Rebuild the client’s trust in the agency and the product team
Lead the newly formed team and give them time to evolve
Coach team members that weren’t on the required level yet
Set up automated quality standards, so the team could shine, with or without me
Be the go-to contact for the client and other teams, so the development process and communication doesn’t dip
In other words, I had to elevate the entire team through ownership, clarity, and influence. I had to become a force multiplier.
Force multipliers don’t just do an excellent job themselves. They raise the bar of the team, other teams, and ultimately the entire company.
They establish systems that run smoothly without them, and ironically, that makes them irreplaceable.
In The Force Multiplier section, we cover how you can become that kind of data engineer. We’ll talk about:
Leadership (no you don’t have to be an extrovert)
Communication - written, spoken, async
Process optimization
Ownership
etc.
These high-leverage skills will help you advance in your career and become an asset to each company you choose to work for.
Become the master of modern data engineering
Mastering the three pillars of modern data engineering will transform you into a more strategic and high-impact data engineer.
Your technical skills might get you in the room, but the combination of all three pillars is what makes people want to keep you there. Even better, it’ll invite you to more important rooms.
Even if you focus on just one pillar, you’ll level up. You work gets cleaner and your impact becomes more visible.
Master two? Now you’re in high-demand with the freedom to choose where and how you work.
But mastering all three will not only be a lifelong journey, but also the ultimate realization of your full potential.
You’ll:
Transform how you’re seen by teams, leaders, and stakeholders
Become highly sought-after across industries and company sizes
Lead by example and be ready to step into official leadership roles and shine
Work on meaningful, high-impact projects that challenge and excite you because that’s what you’ve built for
And most importantly: Actually enjoy every data engineering role you take
This isn’t just about becoming a better data engineer. It’s about becoming the kind of engineer modern companies can’t afford to lose.
Who am I anyway?
Before I moved into data engineering, I spent over 8 years working as a freelance software engineer.
I’ve worked with big industrial clients, leading e-commerce companies, and ambitious startups.
But now you may wonder:
“If most of your career was outside data engineering, what makes you capable of talking about this stuff?”
Because I’ve faced every single one of these problems firsthand.
I lacked technical excellence and built solutions that caused more harm than good
I spent two years on a project that got shut down overnight when COVID hit because there was zero visibility (and also no users)
And yes, I was once the kind of teammate who was easily replaceable and had no impact
Luckily, I learnt from all these mistakes and continued to grow throughout my career.
After deep conversations with several seasoned data engineers, thought-leaders, Heads of Data (Engineering), C-level people, and data architects, I noticed:
The same challenges apply to data engineering, the same lessons matter, and the same transformation is possible.
I’m convinced our profession is evolving and we need to raise the bar together to meet the new requirements.
I may not be the most experienced data engineer, but I’m here to grow alongside you, and share everything I learn along the way.
Your next steps
To get started, just subscribe to the free newsletter.
You’ll get new articles that help you grow as a modern data engineer
If you want to go one step further, there are paid options that offer even more content, templates, checklists, and in the future group calls.
Whatever, you choose, I’m happy you are here.
Let’s build something better, together.