“Having a solid team makes the experience a lot easier for the people receiving and the people building the platform,” said Ryan Cook, senior principal software engineer at Red Hat.

Luca Galante, VP of product and growth at IDP company Humanitec and organizer of PlatformCon, believes that one of those important skills is the ability to have a product mindset, approaching things from a continuous development perspective based on a tight feedback loop with the teams they are building the platforms for, rather than building and shipping software and then being done with it.  

RELATED: IDPs may be how we solve the development complexity problem

“It’s really about seeing developers in a different light, which is the internal customers, and we’re serving them and solving their pain points,” Galante said. 

Cook agrees with that, adding “understanding what the teams need, what the people building the platforms need, is the best way to be successful.”

Communication is also key, because platforms interact with everything — and multiple teams — in an engineering organization. This includes developers, infrastructure and operations (I&O) teams, security teams, architects, executives and more.

“In order for everybody to be on board, there needs to be a driving internal marketer on the platform team that effectively aligns the development of the platform and the benefits that it drives to the vested interests of the different stakeholder groups,” Galante explained. 

For instance, a development team that is experiencing long waits from the infrastructure team could be sold on a platform by being told it’s going to reduce wait times and improve developer experience. It could be sold to the security team as something that is going to enforce governance and policy by default. And it could be sold to the infrastructure team as something that is going to reduce the need to do manual configurations every time a developer needs something. 

Thus, there needs to be someone on the platform engineering team who is able to articulate and communicate these benefits to the various stakeholders, so that everyone understands this is a worthwhile endeavor. 

A third important skill is deep technical capability and understanding, said Zohar Einy, CEO of Port, another IDP provider. He explained that it’s important for a platform engineer to have an understanding of how the components of the company’s technical stack are set up, what development tools are being used, and so on.

“They need to have a very good understanding on how things are wired and how the platform is built behind the scenes,” he said. 

Red Hat’s Cook believes it’s a good idea to have different people with different areas of expertise, like someone that’s really good at telemetry or security, or development or virtualization – or whatever it may be. 

“We all have this unique expertise, but the same goal, and I feel that expertise helps because it gives the ones that are experts in their space the confidence to continue to be experts there, while it gives the other folks breathing room that they don’t have to become experts outside of their individual realms. So everybody kind of leans on each other, which creates a good, friendly relationship internally with the team,” he said. 

Specific roles that make up a platform team

According to Galante, there are four main roles that all platform teams should have: head of platform, platform product manager, platform engineers, and infrastructure platform engineers. 

The head of platform is ultimately the person that is going to motivate and sell the platform to higher-ups in legal and compliance, finance and the executive suite. They are responsible for explaining the value that platforms can have, and to “make sure that they see the platform as a value driver, as opposed to a cost center.” 

They will also continuously update those stakeholders on the progress throughout the platform’s life cycle.

The platform product manager is the person responsible for making sure that the platform is actually made. They’re also there to facilitate compromise for the different stakeholders, like making sure that the security team is happy because security is enforced by the platform or that the architects are happy because the platform fits within the broader enterprise architecture.

They are also responsible for making sure that the end users — the developers — are happy with the platform and actually want to use it. According to Galante, there is a fine line between abstracting away the underlying complexity of the infrastructure while also keeping enough context for developers to do their jobs properly. 

“You want to provide developers with paved roads and really intuitive ways of interacting with your increasingly complex tool chain … But at the end of the day, they’re still engineers. They want to be able to still have some level of control and context around the work that they’re doing. And so that’s what the platform product manager is really focused on,” said Galante.

The final two roles are the platform engineers and infrastructure platform engineers. The reason for the differentiation is that platform engineers are the voice of the developers they’re building for, while infrastructure platform engineers are the voice of the I&O team. 

According to Galante, there can often be so much focus put on improving developer experience, but it’s important to make sure that the needs of the I&O team are also being considered. 

“You can think of the platform essentially as a vending machine that you’re maintaining, growing, and providing as a service to the rest of the organization,” he said. “And so that is where it’s very important to have this kind of role of the infrastructure platform engineer that oftentimes can come from the infrastructure scene and build that bridge to make sure that both perspectives are represented on the platform team.” 

The job types that transition well into platform teams

Einy believes many existing roles can transition well into the platform engineering team, such as DevOps engineers, technical product managers, and SREs.  

According to Einy, DevOps is a spectrum, and there may be DevOps engineers who are more infrastructure oriented and ones that are more experience oriented. He believes that the ones who were responsible for the user-facing processes can translate well into a platform engineer. 

“In the past, the platform engineering responsibility was part of the DevOps responsibility, but now it’s like it went to an entire role of its own,” Port said. 

Cook added that DevOps has likely felt the pain of what it takes to release and maintain software, so they can bring what they’ve learned to the table. 

Einy believes that technical product managers would also do well on a platform engineering team, because they are used to needing to have deep technical knowledge of their products.

And finally, SREs translate well into platform engineers because they’re responsible for quality, making sure that the MTTR is low, and improving the overall resiliency of an organization.

“One of the main values for platform engineering is to create the standards and to maintain the resiliency and efficiency of things,” Einy said.

Now that a team is in place, what’s next? 

Once the platform engineering team is established, it’s important to have strong collaboration within the team, and also with the stakeholders they are building for. In terms of building a good platform engineering culture, Cook recommends establishing an environment where the engineers are respectful of each other and of each other’s time. 

He also added that by bringing in different experts to the team, they will by nature start to depend on each other and get to know each other. “Having those smaller teams with the expertise kind of helps on the friction side, because they’re in it together,” he said.

When it comes to collaborating with the different stakeholders that the platforms are being built for, that platform-as-a-product mindset comes back into play. This collaboration should be a continuous loop, not a one-and-done approach.

According to Einy, platform engineering teams should be conducting surveys, which means they need to know how to run a good survey, which entails knowing what questions to ask, setting goals for the responses, and then finally being able to digest and understand the results. 

He added that it’s also good to be doing data analysis on usage of the platform, who is using it, what parts are getting used, how often it’s used, etc. 

“Again, talking with the people, not in a structured way, but creating some kind of closed group of people that can represent the wider audience and collecting feedback from the field,” Einy said. “I think that these are the things that they need to do continuously to know that they are solving the right problem for the organization.”

Cook added that when he started working at Red Hat, they hosted a “complaint fest” where the development teams came to them and let them know what was wrong in an open, constructive way. He said that developers were a bit hesitant to speak up at first, but once one person started the discussion, that broke the ice for the rest of the team to be open with what’s wrong. 

“If you can let everybody know that you do really care about the concerns and you are trying to fix them, they’re going to be a lot more willing to use your product than if you just do it without them,” Cook explained.

The post Building a platform engineering team that’s set up for success appeared first on SD Times.

This has led to a lot of tool sprawl in development environments. Developers need an IDE to write the code, an incident management platform to fix issues, a platform for managing the infrastructure they are deploying to, an observability platform to track performance, and more. And sometimes, developers are reliant on people in other areas of the business to tackle some of those responsibilities, which can slow things down.

Out of this complexity, in the past few years a new type of tool has emerged to help developers get a better handle on all of these different things: the Internal Developer Portal (IDP). 

“Additional responsibilities are getting pushed to developers, and so they have just more and more on their plates, in architectures that are more and more complex, and a tool chain that they have to use that is, again, just more and more complex, fragmented,” said John Laban, CEO of OpsLevel, a provider of IDP software.

According to Zohar Einy, CEO of the IDP company Port, an IDP is a layer on top of all other applications that gives developers easy access to all the different components and information they need to do their jobs. 

And the technology is really taking off; Gartner predicts that by 2026, 80% of large software companies will have platform engineering teams, which are teams that create the IDPs. 

To give a practical example of what an IDP looks like in practice, an IDP could be connected up to Datadog to display key metrics through a widget or dashboard. It doesn’t provide actual access to observability features, but if the developer needs to dive deeper, the IDP provides a clear path into Datadog to leverage the specialties of that platform. 

“A developer portal is not a replacement for all the tools, but it’s a front door,” said Einy. “It gives you the most important things that you need to consume from all the other tooling, showing just the most important data with context, just the most important actions inside the portal. But from there, you can dive deeper into the different tools that specialize in the specific area.”

Essentially it provides a contextual place for all of those different tools, and then you can go from the portal to all these different paths. 

The three pillars of an IDP

According to Laban, there are three pillars to IDPs: visibility, standards, and self-service. Visibility is all about getting a handle on the complexity of the software development toolchain, both in understanding what’s out there and in how those tools interoperate with each other. This includes knowing about their APIs, documentation, who owns them, their dependencies, and more, he explained.

“And of course, you need to know where to find the metrics, dashboards, the logs, all the different things that you might need to operate each of these services,” said Laban.

He explained that most organizations today don’t really have a strong handle on that, or if they do, the information is stored in places like a spreadsheet or wiki page.

The second pillar is standards. “Once you do have that understanding of what’s out there, the next question is – okay, we have all these different engineering teams, they are all building software somewhat differently, and it’s released independently and autonomously. In our current model of how we work, are we all building software the right way across the organization?”

According to Laban, IDPs provide a way of visualizing these standards and measuring them across all services. 

And finally, the third IDP pillar is self-service, which is about making it easier for developers to get what they need in order to build their software. 

“With that comes templating and scaffolding for new service creation, being able to generate new services, checking repos, using all the up-to-date libraries, as well as providing easy self-service actions for developers. So rather than having to create tickets and wait on other teams, they can ideally do things for themselves, whether it’s provisioning hardware or environments or even setting the service up in other tooling,” said Laban. 

Advice for getting started

Kenneth Rose, co-founder and CTO at OpsLevel, believes that one way to ensure success is to start gradually by picking one problem and solving it rather than getting too excited and trying to connect everything up at the beginning. “The challenge there is that it’s kind of too much, and you end up spreading yourself really thin,” said Rose. “Where we’ve seen customers be most successful is to look first at the catalog setup, then we can talk about standards, and once we have this foundation of a catalog that we know is accurate and up-to-date, then we can also focus on developer self-service.”

Einy believes that successful IDP implementation can be achieved by viewing it as an Agile process versus a Waterfall one where once it’s set up you never adjust it. 

“What you have to do is to treat the portal as a product, and to implement the portal in phases and make it an Agile process,” he said. “So you implement an MVP, you roll it out to the end users, you collect feedback, and then you reiterate, you create another version, get feedback, you make mistakes, you fix, you improve.” 

It’s also not a one-size-fits-all thing, so what works for one company might not work for another. For instance, Netflix and Citibank may both have an IDP, but those are two very different companies with very different tech stacks and very different needs, so their IDPs will also look very different, Einy explained.

“Netflix might allow developers to do one click and to change the production for the entire customer base. At Citibank it’s more of a restrictive process because it’s highly regulated and there are approvals to be made, and there is a different DNA of what it means to be an engineer for such a company,” said Einy.

Different companies may also have different stakeholders utilizing the IDP so it’s important to ensure that the portal is dynamic enough to meet the needs of all stakeholders, he said. 

He also believes being dynamic in the sense of scalability is important so that the IDP you have today can evolve into the IDP you need in the future.  

“You need to make sure that you choose a solution that allows you to face the test of time, and meet your current needs, your current DNA, and can go with you and not hold back the innovation for the company, but rather to dictate the innovation and push everyone forward,” said Port.

And finally, Rose said that another key to success is to leverage automation when it comes to cataloging and mapping out the services, which can help speed up the implementation timeline.

You may also like…

Analyst View: What’s new, what’s now, and what’s next in platform engineering

The challenges with platform engineering don’t have to do with engineering

The post IDPs may be how we solve the development complexity problem appeared first on SD Times.

The new offering will enable development teams to more easily get set up with their own internal developer portal using Backstage. It includes a Setup Wizard to install Portal and link it to a GitHub organization and cloud provider; a Catalog Wizard to import services, websites, and libraries from GitHub Enterprise; and onboarding guides that explain how to create Software Templates, import docs, and more. 

Software Templates provide reusable building blocks for backend services, websites, etc. These will help facilitate easier setup for future projects, Spotify explained. 

The Software Catalog is one of the central components of Portal and it is where all the development team’s services, websites, and libraries can be found. According to Spotify, the Catalog provides insight into who owns what projects, makes it easier to discover components and dependencies related to those projects, and cuts down on the time spent finding projects. 

Portal also offers a number of plugins that can be installed to make it more customizable, such as Spotify’s Soundcheck plugin which can be used to implement more quality checks.

According to Zohar Einy, CEO of the internal development platform Port, this move is an indication that Spotify is taking its development tool business more seriously and is branching out from just being a provider of an open source project to becoming a vendor of software development tools. 

“Ideally, developer portals should offer a product-like experience, delivering a comprehensive portal that meets all developer needs and creating a standardized way of work,” he said. “While the product aspect of this announcement is intriguing, the real story is Spotify’s business play. This could alter how Backstage’s role is perceived in the internal developer portal community.”

The post Spotify launches Portal for Backstage to simplify process of getting developer portals setup appeared first on SD Times.

What is new is the solution: platform engineering. Operating platforms sit between the end user and the backing services on which they rely. The platform is an internal software product, built by a dedicated team, that provides a curated collection of reusable components, tools, services, and knowledge, packaged for easy consumption. As illustrated below, the platform becomes a layer of abstraction between the developer and the messy complexities of operations.

The specific components and capabilities of each platform vary widely. Ultimately, the platform is whatever a development team needs it to be. The platform team’s job is to build that product. Each platform may be unique, but all platforms are:

• Productized: The platform is a product. User feedback directs product strategy.
• User-centric: Platforms solve users’ problems, not operators’. For example, developers probably don’t want a fast way to build a VM; they don’t want to think about infrastructure at all.
• Self-service: Developers can access everything they need from a single source, without opening tickets, sending emails, or filing requests.
• Consistent and compliant: Standards are built in, so that users cannot deliver code that is out-of-spec or insecure.

The platform is a “paved road” including both guidelines (recommended ways of travelling) and guardrails (hard boundaries the user can’t cross). A platform might enforce compliance guardrails: “You must run these automated tests of your security posture before deploying.” However, it might only suggest certain workflows: “We recommend the following tool for these use cases.”

What’s New: Platform Engineering Fulfills the Promises of DevOps

It’s important to distinguish platform engineering from what has come before. Automation is nothing new; nor are calls for better collaboration between developers and operators. A platform goes beyond existing techniques and tools. It is a new software product, with its own customers, lifecycle, user contracts, and lofty expectations.

Platform engineering represents the state of the art in DevOps. Not surprisingly, therefore, platform engineering has quickly become the hottest topic of conversation in that world, spawning its own user community and conferences. Gartner named platform engineering a Top Strategic Technology Trend in both 2023 and 2024 and predicts that, by 2026, 80% of large software engineering organizations will establish platform engineering teams.

What’s Now: The Platform Revolution Has Begun

IT shops have already begun to implement platform teams. Most start by building an internal developer portal, often using the open-source project Backstage. The portal is a central service catalog and document repository. It can also be a graphical user interface for automations and delivery pipelines. The user experience should be markedly better than whatever homebrew solutions developers have built for themselves. The goal is not to force developers onboard; they should want to use it.

Platforms make developers more productive. They free developers from the burden of building out their own operating environments. This allows developers to avoid unnecessary “glue” work and focus on writing code that creates value. When measuring the value of the platform—or making a business case to build one—focus on its positive impact on productivity. Deployment rates should increase; error rates, incidents, exceptions, rework, and time-to-value should all decrease. 

What’s Next: Platforms Expand and Evangelize

Platforms start small, often with only documentation and a service catalog. But even this is valuable. It saves developers from having to open tickets or send emails. Over time, the platform grows more capable. In the maximal vision of platform engineering, the platform has its own set of APIs. Developers write to the platform, and its abstractions become load-bearing parts of the application.

The platform can also expand to other users—data scientists, for example, or even business units looking to automate their work. They too will find value in a platform that meets users where they are. A platform team that can provide building blocks that are immediately useful, at an appropriate cognitive load, without unduly constraining users or forcing them into foreign ways of working, provides value now and in the future.

The post Analyst View: What’s new, what’s now, and what’s next in platform engineering appeared first on SD Times.

But wait, how many Kubernetes experts do we have? We’ve got an industry-wide shortage of skilled software development and operations talent—and the complexity of cloud native development is exacerbating the problem. We’re not going to hire our way out of this mess!

Skill shortages stymie cloud native innovation

Even non-technical executives now understand the basic benefits of cloud native software. They know it has something to do with Kubernetes pushing out containers, so the resulting applications are more modular and take advantage of elastic cloud infrastructures.

There’s way more to it than that. The cloud native landscape is a beehive of open-source projects for configuration, networking, security, data handling, service mesh – at various maturity stages. There are also hundreds of vendors offering their development, management and support tools atop this ever-moving CN raft.

A developer’s learning curve is steep and continuous, since this year’s stack might no longer be relevant next year. Outsourcing is tough too, when the few consultants with a real knack for cloud native development are busy and expensive.

The only way to sustainably grow is through building cloud native development talent and capabilities from within, following the introduction and maturation of the space, as well as keeping tabs on the vendor and end user community at large.

While the market demand for digital offerings keeps doubling every couple years, IT budgets often get a budget belt-tightening as a reward for managing and scaling so much technology.

Smaller and leaner teams are expected to deliver twice the output with half the people. The need for specialized skill positions only increases as concepts like event-driven architecture, data lakehouses, real-time analytics, and zero-trust security policies turn into production-grade requirements. 

Why platform engineering matters

No matter what target environment they are contributing to, developers still spend most of their time coding within an IDE. Over the years, vendors have tried everything from low-code tools to process toolkits to lower the skills bar, but the pipelines don’t translate into easy wizards. 

Complex open-source tooling, third party service APIs, and code that is being mixed and matched from GitOps-style repos are driving cloud native development teams toward a new platform engineering approach.

Platform engineering practices seek to create shared resources for development environments–encouraging code, component, and configuration reuse. 

Common platform engineering environments can be represented within a self-service internal development portal or an external partner marketplace, often accompanied by concierge-style support or advisory services from an expert team that curates and reviews all elements in the platform.

It’s critical to govern the platform’s self-service policies for access permissions, code, logic, data, and automation at just the right level of control for the business it supports. 

Too much flexibility ends up creating overhead, as diverse stakeholders fail to distinguish between the relative value or usefulness of so many unvalidated and poorly categorized components of the platform.

Too much rigidity in policy design can create the opposite problem, where centralized governance and approval cycles for every element slow down solution delivery and take away the innovative freedom developers crave.

A modern approach to cloud native platform engineering can finally bring the principles of governance, consistency and reuse to the table.

Speedy innovation through infrastructure abstraction

Refreshingly—or maddeningly—there’s no single right way to ‘do’ cloud native. Even Kubernetes is by itself just an abstraction of container orchestration and only one option for going cloud native.

As an open-source movement, the CNCF purposely leaves the future approach open to interpretation by the community. It doesn’t dictate a particular language, or even a specific piece of infrastructure. 

That’s excellent, but it also leaves short-handed dev teams managing complex plumbing and experimenting with integration options, rather than building better functionality. That’s where platform engineering practices can save the day.

The decision to create a platform is a commitment to help developers of varying skill levels abstract away the complexity of underlying cloud native architectures with interfaces and tools atop readily configured environments.

A platform engineering approach must offer ease of use, elimination of toil, and reduced cognitive load for development teams—helping orgs attract and retain the best talent.

The Intellyx Take

Truly innovative ideas that impact customers represent a core competitive differentiator, and should grow from within the enterprise. That’s difficult when the supply of skilled cloud native development resources is constrained.

Fortunately, platform engineering organizations and technology stacks can automate some of the most difficult work of delivering on the needs of API-driven microservices environments.

The post Patch the cloud native development talent gap with platform engineering appeared first on SD Times.

“The Spotify Plugins for Backstage bundle is the next step toward Spotify’s goal to share what we’ve learned with the world. We’re confident these plugins will take each and every developer portal built on Backstage to the next level, by making developers in your organization happier and more productive,” Austin Lamon, Director, GM of Backstage at Spotify wrote in a blog post. 

The first plugin, Soundcheck, codifies engineering best practices to improve quality, reliability, security, and alignment throughout a software ecosystem in a gamified way. It includes an entity page that  provides a comprehensive snapshot of a specific entity’s tech health, pass or fail checks, and an overview page that provides a grid view of all entities owned by specific groups inside the organization.

The next plugin, Role-Based Access Control helps users manage access and protect their data in Backstage to meet evolving security and compliance needs. 

Skill Exchange is an internal marketplace to promote and seek out unique, on-the-job learning opportunities for developers and other members of the tech ecosystem. It offers search and quick navigation to learning opportunities. 

Pulse lets users track productivity and satisfaction metrics so that they can visualize metrics in Backstage.

Lastly, Insights enables developers to see how the organization is performing in Backstage by underscoring on the roadmap where things need to be doubled down on or deprecated. 

“These plugins have been put through endless hours of internal use, iteration, and improvement. We know they work because they’ve done amazing things for us — from Soundcheck bringing test flakiness to below 1% to Skill Exchange powering thousands of collaborative hacks between teams,” Lamon added. 

The post Spotify is introducing new plugins for Backstage appeared first on SD Times.

The project has its origins at Spotify. In 2016 the company was growing quickly and struggling to onboard engineers fast enough to meet its needs. Creating Backstage enabled Spotify engineers to be more productive and work faster.

After using it internally for a few years, Spotify open-sourced the project in March 2020 and then later that year in September it entered the CNCF Sandbox. 

Since joining the CNCF Backstage has seen growth across its core components and features. Maintainers have focused efforts into updating, refining, documenting, deprecating and stabilizing core components in advance of the 1.0 release of the Core Framework, which is still upcoming. The Core Framework includes the Software Catalog, Software Templates, TechDocs, and API Reference. 

Backstage is currently in use publicly at over 100 companies, including American Airlines, HelloFresh, Netflix, Wayfair, and more. 

“It’s amazing to see organizations from vastly different industries adopting Backstage as their development platform and working together at making the developer experience better for everyone,” said Johan Haals, senior site reliability engineer at Spotify. “Our community has grown tremendously this past year, and I’m excited for the efforts the community is bringing to increase the project’s velocity.”  

The post CNCF accepts Backstage as incubating project appeared first on SD Times.