Hey Guys,
If the Modern Data Stack (MDS) is reshaping Data Engineering, Platform Engineering is becoming a newer and valuable role.
How do you become a platform engineer?
Platform engineers need a bachelor's degree in computer science or an associate's degree and relevant work experience.
...
The most similar professions to Platform Engineer are:
Software Engineer.
Software Developer.
Devops Engineer.
Cloud Engineer.
A platform engineer is responsible for the design, implementation and maintenance of a company’s technology infrastructure. They are often tasked with ensuring that all systems are running smoothly and resolving any issues that may arise.
In a sense this is why being a Data Engineering is becoming a better definition of a success role than being a Data scientist itself. Data infrastructure as a service is quickly evolving.
The Modern Data Stack is itself evolving as well to suit the supply-demands of organizations. A modern data stack is a solution that can help an organization save time, effort, and money. It is faster, more scalable, and more accessible than the traditional data stack.
Digital Transformation in the Data Age
One way to gain a competitive advantage is to have an efficient mechanism for deploying & operating applications in production that enables teams to provide features to end-users faster.
To achieve that, many companies realized that it helps to have a dedicated team that builds & maintains a common platform and constantly looks to innovate how applications are deployed and operated in Production. This practice of creating a common platform as a product is called Platform Engineering.
Platform Engineering is thus said to be that process of selecting frameworks -- i.e. whether to use third-party frameworks, rather than making your own, while also showing how to tie the frameworks together to build a platform that applications can be built on.
The rise of microservices, container orchestration, and the like have introduced novel engineering challenges. Platform engineering teams have formed at a number of organizations to shoulder these responsibilities.
At the same time, cloud-native organizations have radically changed how they’re organized, moving from large departments (development, QA, operations, release) to smaller, independent development teams.
Rise of Site Reliability Engineering (SRE) and Platform Engineering
SRE and platform engineering are spiritual successor of traditional operations teams, and bring the discipline of software engineering to different aspects of operations.
Site Reliability Engineering and Platform Engineering
Platform engineering teams apply software engineering principles to accelerate software delivery. Platform engineers ensure application development teams are productive in all aspects of the software delivery lifecycle.
Site reliability engineering teams apply software engineering principles to improve reliability. Site reliability engineers minimize the frequency and impact of failures that can impact the overall reliability of a cloud application.
There is truth in noting that the title, “Platform Engineer” is nothing but a new title. However, a number of factors are, and continue to, cause the traditional responsibilities of a Site Reliability Engineer (SRE) to shift.
New Principles:
Platform as product.
Self-service platforms.
A constant focus on eliminating toil. As defined in the Google SRE book, toil is manual, repetitive, automatable, tactical work. The best SRE and platform teams identify toil, and work to eliminate it.
Therefore the evolution of the Modern Data Stack entails news ways software development engineers, data engineers and data science teams work together.
So, the role, “Platform Engineer” is considered by some to just be a different title for a job that has been traditionally performed by an infrastructure team.
Just as data engineers and data scientists may have a different skill set and slightly different roles, so too might a platform engineer with your average software engineer:
Software engineer
Specialist regarding developing software
Often also called “software developer”Platform engineer
Specialist regarding a certain platform, stack, etc…
As a Platform Engineer you will help to deploy, manage, fix and reinvent the tools, services and components that the network engineers rely on to automate our network and keep it operational.
Deployment and maintenance
Migration and automation
Though what a Platform Engineer is also somewhat subjective. Asking ten engineers this same question would likely yield ten different answers.
Think about it, Platform Engineers are applied specialists. Platform engineers commonly work in an IT department or as part of a larger engineering team. Their job requires them to have a deep understanding of computer hardware and software—they must be able to identify and resolve problems quickly and efficiently.
Roles and Job Duties of the Platform Engineer
Platform engineers have a wide range of responsibilities, which can include:
Designing and implementing an automation framework for all aspects of the application lifecycle, including build, test, and deployment
Performing operations tasks such as monitoring application performance over time or troubleshooting issues with production applications
Monitoring network performance to ensure that the platform can handle increased traffic from new applications or services
Designing, building, and maintaining the systems underlying an application platform
Developing new technologies to support existing applications or creating new applications using new technologies
Participating in meetings with executives to discuss technical issues and propose solutions
Reviewing code written by other engineers to ensure it meets standards for quality, security, and performance
Collaborating with other members of the engineering team to design new features or improve existing ones
Consuming and analyzing large amounts of data in order to identify patterns and make predictions about future trends
Compensation
Source: Glassdoor
Platform Engineer Salary & Outlook
The salary of a platform engineer can vary depending on their level of education, years of experience, and the company they work for. They may also earn additional compensation in the form of bonuses or commissions.
Median Annual Salary: $125,000 ($60.1/hour)
Top 10% Annual Salary: $195,000 ($93.75/hour)
The employment of platform engineers is expected to grow much faster than average over the next decade.
Many aspiring platform engineers will receive on-the-job training from their new employer after they are hired.
Core Platform Engineer Skills
Platform engineers need the following skills in order to be successful:
Coding:
Coding is the process of creating software. As a platform engineer, you may be responsible for creating the software that supports the company’s cloud infrastructure. Coding skills can help you develop the software that keeps the company’s cloud infrastructure running smoothly.
Debugging:
Debugging is the process of finding and resolving errors in computer code. As a platform engineer, you may be responsible for identifying and resolving errors in the software of a company’s website or an app. Knowing how to debug effectively can help you save time and money.
Hardware knowledge:
Hardware knowledge is the ability to understand the components of a device and how they work together. This is an important skill for a platform engineer because it allows them to troubleshoot and repair computer systems. It’s also important for them to understand how software interacts with hardware.
Networking:
As an engineer, you may need to collaborate with other professionals in your field. Having strong networking skills can help you form connections with others in your industry. You can use your networking skills to find potential job opportunities, find other professionals to ask for advice or find other people who can help you learn more about your field.
Project management:
Project management skills can be helpful for engineers who work on large projects. This can include planning, organizing and overseeing the completion of a project. Project management skills can help you delegate tasks, set deadlines and track the progress of a project.
Platform development such as frameworks, operating system/kernel, Programming languages, Compilers etc demands considerable software skill. I hope this quick overview gives you a more realistic idea what Platform Engineering is and what it entails.
An organization’s platform engineers tailor infrastructure as code tools to the needs of the organization’s application developers.
Infrastructure as code
Infrastructure as code is one of a number of factors that have helped boost the notion of a platform engineer within the collective conscience.
Gartner expects that by 2026, 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components and tools for application delivery.
Platform Engineering
Platform engineers constantly examine the entire software development lifecycle from source to production. From this introspective process, they build a workflow that enables application developers to rapidly code and ship software.
Site Reliability Engineering
Site reliability engineers create and evolve systems to automatically run applications, reliably. The concept of site reliability engineering originated at Google, and is documented in detail in the Google SRE Book.
Furthermore both DevOps and GitOps are a loosely codified set of principles of how to manage different aspects of infrastructure. The core principles of both of these philosophies -- automation, infrastructure as code, application of software engineering -- are very similar.
If you think of ROI ultimately Platform engineering is an emerging technology approach that can accelerate the delivery of applications and the pace at which they produce business value.
When we study software development, we might not have an inkling we might end up as a “Platform Engineer”. Yet times are changing. Platform engineering is an emerging trend intended to modernize enterprise software delivery, particularly for digital transformation. The engineering platform is created and maintained by a dedicated product team, designed to support the needs of software developers and others by providing common, reusable tools and capabilities, and interfacing to complex infrastructure.
Read: What is the Modern Data Stack?
Thanks for reading! Give this post a like or a quick comment if you found the topic useful. I’m slowing getting into more data science topics on this Newsletter.
Thanks!