According to Jonathan Harding, Senior Product Manager of Value Stream Management at HCLSoftware, in an era where organizations have become so focused on how to be more productive, this discipline has gained popularity because “it gets new employees productive quickly, and it gets existing employees able to deliver quickly and in a way that is relatively self-sufficient.”

Platform engineering teams work to build an internal developer portal (IDP), which is a self-service platform that developers can use to make certain parts of their job easier. For example, rather than a developer needing to contact IT and waiting for them to provision infrastructure, that developer would interact with the IDP to get that infrastructure provisioned.

Essentially, an IDP is a technical implementation of a DevOps objective, explained Chris Haggan, Head of HCL DevOps at HCLSoftware.

“DevOps is about collaboration and agility of thinking, and platform engineering is the implementation of products like HCL DevOps that enable that technical delivery aspect,” Haggan said.

Haggan looks at platform engineering from the perspective of having a general strategy and then bringing in elements of DevOps to provide a holistic view of that objective. 

“I want to get this idea that a customer has given me out of the ideas bucket and into production as quickly as I can. And how do I do that? Well, some of that is going to be about the process, the methodology, and the ways of working to get that idea quickly through the delivery lifecycle, and some of that is going to be about having a technical platform that underpins that,” said Haggan. 

IDPs typically include several different functionalities and toolchains, acting as a one-stop shop for everything a developer might need. From a single platform, they might be able to create infrastructure, handle observability, or set up new development environments. The benefits are similar in HCL DevOps, but by coming in a ready-to-use, customizable package, development teams don’t have to go through the process of developing the IDP and can skip right to the benefits. 

Haggan explained that the costs of building and maintaining a platform engineering system are not inconsequential. For instance, they need to integrate multiple software delivery systems and figure out where to store metrics, SDLC events, and other data, which often requires setup and administration of a new database. 

Plus, sometimes teams design a software delivery system that incorporates their own culture nuances, which can sometimes be helpful, but other times “they reflect unnecessary cultural debt that has accumulated within an organization for years,” said Haggan.

HCL DevOps consists of multifaceted solutions, with the three most popular being:

• HCL DevOps Test: An automated testing platform that covers UI, API, and performance testing, and provides testing capabilities like virtual services and test data creation.
• HCL DevOps Deploy: A fully automated CI/CD solution that supports a variety of architectures, including distributed multi-tier, mobile, mainframe, and microservices. 
• HCL DevOps Velocity: The company’s value stream management offering that pulls in data from across the SDLC to provide development teams with useful insights.

Haggan admitted that he’s fully aware that organizations will want to customize and add new capabilities, so it’s never going to be just their platform that’s in play. But the benefit they can provide is that customers can use HCL DevOps as a starting point and then build from there. 

“We’re trying to be incredibly open as an offering and allow customers to take advantage of the tools that they have,” Haggan said. “We’re not saying you have to work only with us. We’re fully aware that organizations have their own existing workflows, and we’re going to work with that.”

To that end, HCL offers plugins that connect with other software. For instance, HCL DevOps Deploy currently has about 200 different plugins that could be used, and customers can also create their own, Harding explained. 

The plugin catalog is curated by the HCL DevOps technical team, but also has contributions from the community submitted through GitHub. 

Making context switching less disruptive

Another key benefit of IDPs is that they can cut down on context switching, which is when a developer needs to switch to different apps for different tasks, ultimately taking them out of their productive flow state.  

“Distraction for any knowledge worker in a large enterprise is incredibly costly for the enterprise,” said Harding. “So, focus is important. I think for us, platform engineering — and our platform in general — allows a developer to stay focused on what they’re doing.”

“Context switching will always be needed to some degree,” Haggan went on to say. A developer is never going to be able to sit down for the day and not ever have to change what they’re thinking about or doing. 

“It’s about making it easy to make those transitions and making it simple, so that when I move from planning the work that I’m going to be doing to deploying something or testing something or seeing where it is in the value stream, that feels natural and logical,” Haggan said. 

Harding added that they’ve worked hard to make it easy to navigate between the different parts of the platform so that the user feels like it’s all part of the same overall solution. That aspect ultimately keeps them in that same mental state as best as possible.

The HCL DevOps team has designed the solution with personas in mind. In other words, thinking about the different tasks that a particular role might need to switch between throughout the day.

For instance, a quality engineer using a test-driven development approach might start with writing encoded acceptance criteria in a work-item management platform, then move to a CI/CD system to view the results of an automated test, and then move to a test management system to incorporate their test script into a regression suite. 

These tasks span multiple systems, and each system often has its own role-based access control (RBAC), tracking numbers, and user interfaces, which can make the process confusing and time-consuming, Haggan explained. 

“We try to make that more seamless, and tighten that integration across the platform,” said Harding. “I think that’s been a focus area, really looking from the end user’s perspective, how do we tighten the integration based on what they’re trying to accomplish?”

To learn more about how HCL DevOps can help achieve your platform goals and improve development team productivity, visit the website to book a demo and learn about the many capabilities the platform has to offer. 

The post HCL DevOps streamlines development processes with its platform-centric approach appeared first on SD Times.

“Having a solid team makes the experience a lot easier for the people receiving and the people building the platform,” said Ryan Cook, senior principal software engineer at Red Hat.

Luca Galante, VP of product and growth at IDP company Humanitec and organizer of PlatformCon, believes that one of those important skills is the ability to have a product mindset, approaching things from a continuous development perspective based on a tight feedback loop with the teams they are building the platforms for, rather than building and shipping software and then being done with it.  

RELATED: IDPs may be how we solve the development complexity problem

“It’s really about seeing developers in a different light, which is the internal customers, and we’re serving them and solving their pain points,” Galante said. 

Cook agrees with that, adding “understanding what the teams need, what the people building the platforms need, is the best way to be successful.”

Communication is also key, because platforms interact with everything — and multiple teams — in an engineering organization. This includes developers, infrastructure and operations (I&O) teams, security teams, architects, executives and more.

“In order for everybody to be on board, there needs to be a driving internal marketer on the platform team that effectively aligns the development of the platform and the benefits that it drives to the vested interests of the different stakeholder groups,” Galante explained. 

For instance, a development team that is experiencing long waits from the infrastructure team could be sold on a platform by being told it’s going to reduce wait times and improve developer experience. It could be sold to the security team as something that is going to enforce governance and policy by default. And it could be sold to the infrastructure team as something that is going to reduce the need to do manual configurations every time a developer needs something. 

Thus, there needs to be someone on the platform engineering team who is able to articulate and communicate these benefits to the various stakeholders, so that everyone understands this is a worthwhile endeavor. 

A third important skill is deep technical capability and understanding, said Zohar Einy, CEO of Port, another IDP provider. He explained that it’s important for a platform engineer to have an understanding of how the components of the company’s technical stack are set up, what development tools are being used, and so on.

“They need to have a very good understanding on how things are wired and how the platform is built behind the scenes,” he said. 

Red Hat’s Cook believes it’s a good idea to have different people with different areas of expertise, like someone that’s really good at telemetry or security, or development or virtualization – or whatever it may be. 

“We all have this unique expertise, but the same goal, and I feel that expertise helps because it gives the ones that are experts in their space the confidence to continue to be experts there, while it gives the other folks breathing room that they don’t have to become experts outside of their individual realms. So everybody kind of leans on each other, which creates a good, friendly relationship internally with the team,” he said. 

Specific roles that make up a platform team

According to Galante, there are four main roles that all platform teams should have: head of platform, platform product manager, platform engineers, and infrastructure platform engineers. 

The head of platform is ultimately the person that is going to motivate and sell the platform to higher-ups in legal and compliance, finance and the executive suite. They are responsible for explaining the value that platforms can have, and to “make sure that they see the platform as a value driver, as opposed to a cost center.” 

They will also continuously update those stakeholders on the progress throughout the platform’s life cycle.

The platform product manager is the person responsible for making sure that the platform is actually made. They’re also there to facilitate compromise for the different stakeholders, like making sure that the security team is happy because security is enforced by the platform or that the architects are happy because the platform fits within the broader enterprise architecture.

They are also responsible for making sure that the end users — the developers — are happy with the platform and actually want to use it. According to Galante, there is a fine line between abstracting away the underlying complexity of the infrastructure while also keeping enough context for developers to do their jobs properly. 

“You want to provide developers with paved roads and really intuitive ways of interacting with your increasingly complex tool chain … But at the end of the day, they’re still engineers. They want to be able to still have some level of control and context around the work that they’re doing. And so that’s what the platform product manager is really focused on,” said Galante.

The final two roles are the platform engineers and infrastructure platform engineers. The reason for the differentiation is that platform engineers are the voice of the developers they’re building for, while infrastructure platform engineers are the voice of the I&O team. 

According to Galante, there can often be so much focus put on improving developer experience, but it’s important to make sure that the needs of the I&O team are also being considered. 

“You can think of the platform essentially as a vending machine that you’re maintaining, growing, and providing as a service to the rest of the organization,” he said. “And so that is where it’s very important to have this kind of role of the infrastructure platform engineer that oftentimes can come from the infrastructure scene and build that bridge to make sure that both perspectives are represented on the platform team.” 

The job types that transition well into platform teams

Einy believes many existing roles can transition well into the platform engineering team, such as DevOps engineers, technical product managers, and SREs.  

According to Einy, DevOps is a spectrum, and there may be DevOps engineers who are more infrastructure oriented and ones that are more experience oriented. He believes that the ones who were responsible for the user-facing processes can translate well into a platform engineer. 

“In the past, the platform engineering responsibility was part of the DevOps responsibility, but now it’s like it went to an entire role of its own,” Port said. 

Cook added that DevOps has likely felt the pain of what it takes to release and maintain software, so they can bring what they’ve learned to the table. 

Einy believes that technical product managers would also do well on a platform engineering team, because they are used to needing to have deep technical knowledge of their products.

And finally, SREs translate well into platform engineers because they’re responsible for quality, making sure that the MTTR is low, and improving the overall resiliency of an organization.

“One of the main values for platform engineering is to create the standards and to maintain the resiliency and efficiency of things,” Einy said.

Now that a team is in place, what’s next? 

Once the platform engineering team is established, it’s important to have strong collaboration within the team, and also with the stakeholders they are building for. In terms of building a good platform engineering culture, Cook recommends establishing an environment where the engineers are respectful of each other and of each other’s time. 

He also added that by bringing in different experts to the team, they will by nature start to depend on each other and get to know each other. “Having those smaller teams with the expertise kind of helps on the friction side, because they’re in it together,” he said.

When it comes to collaborating with the different stakeholders that the platforms are being built for, that platform-as-a-product mindset comes back into play. This collaboration should be a continuous loop, not a one-and-done approach.

According to Einy, platform engineering teams should be conducting surveys, which means they need to know how to run a good survey, which entails knowing what questions to ask, setting goals for the responses, and then finally being able to digest and understand the results. 

He added that it’s also good to be doing data analysis on usage of the platform, who is using it, what parts are getting used, how often it’s used, etc. 

“Again, talking with the people, not in a structured way, but creating some kind of closed group of people that can represent the wider audience and collecting feedback from the field,” Einy said. “I think that these are the things that they need to do continuously to know that they are solving the right problem for the organization.”

Cook added that when he started working at Red Hat, they hosted a “complaint fest” where the development teams came to them and let them know what was wrong in an open, constructive way. He said that developers were a bit hesitant to speak up at first, but once one person started the discussion, that broke the ice for the rest of the team to be open with what’s wrong. 

“If you can let everybody know that you do really care about the concerns and you are trying to fix them, they’re going to be a lot more willing to use your product than if you just do it without them,” Cook explained.

The post Building a platform engineering team that’s set up for success appeared first on SD Times.

Then came the GPUs and LLMs.

All of a sudden, enterprises in all industries found themselves in an all-out effort to position AI – both traditional and generative – at the core of as many business processes as possible, with as many employee- and customer-facing AI applications in as many geographies as they can manage concurrently. They’re all trying to get to market ahead of their competitors. Still, most are finding that the informal operational approaches they had been taking to their modest AI initiatives are ill-equipped to support distributed AI at scale.

They need a different approach.

Platform Engineering Must Move Beyond the Application Development Realm

Meanwhile, in DevOps, platform engineering is reaching critical mass. Gartner predicts that 80% of large software engineering organizations will establish platform engineering teams by 2026 – up from 45% in 2022. As organizations scale, platform engineering becomes essential to creating a more efficient, consistent, and scalable process for software development and deployment. It also helps improve overall productivity and creates a better employee experience.

The rise of platform engineering for application development, coinciding with the rise of AI at scale, presents a massive opportunity. A helpful paradigm has already been established: Developers appreciate platform engineering for the simplicity these solutions bring to their jobs, abstracting away the peripheral complexities of provisioning infrastructure, tools, and frameworks they need to assemble their ideal dev environments; operations teams love the automation and efficiencies platform engineering introduces on the ops side of the DevOps equation; and the executive suite is sold on the return the broader organization is seeing on its platform engineering investment.

Potential for similar outcomes exists within the organization’s AI operations (AIOps). Enterprises with mature AIOps can have hundreds of AI models in development and production at any time. In fact, according to a new study of 1,000 IT leaders and practitioners conducted by S&P Global and commissioned by Vultr, each enterprise employing these survey respondents has, on average, 158 AI models in development or production concurrently, and the vast majority of these organizations expect that number to grow very soon.

When bringing AIOps to a global scale, enterprises need an operating model that can provide the agility and resiliency to support such an order of magnitude. Without a tailored approach to AIOps, the risk posed is a perfect storm of inefficiency, delays, and ultimately, the potential loss of revenue, first-market advantages, and even crucial talent due to the impact on the machine learning (ML) engineer experience.

Fortunately, platform engineering can do for AIOps what it already does for traditional DevOps.

The time is now for platform engineering purpose-built for AIOps

Even though platform engineering for DevOps is an established paradigm, a platform engineering solution for AIOps must be purpose-built; enterprises can’t take a platform engineering solution designed for DevOps workflows and retrofit it for AI operations. The requirements of AIOps at scale are vastly different, so the platform engineering solution must be built from the ground up to address those particular needs.

Platform engineering for AIOps must support mature AIOps workflows, which can vary slightly between companies. However, distributed enterprises should deploy a hub-and-spoke operating model that generally comprises the following steps:

• Initial AI model development and training on proprietary company data by a centralized data science team working in an established AI Center of Excellence

• Containerization of proprietary models and storage in private model registries to make all models accessible across the enterprise

• Distribution of models to regional data center locations where local data science teams fine-tune models on local data

• Deployment and monitoring of models to deliver inference in edge environments

In addition to enabling the self-serve provisioning of the infrastructure and tooling preferred by each ML engineer in the AI Center of Excellence and the regional data center locations, platform engineering solutions built for distributed AIOps automate and simplify the workflows of this hub-and-spoke operating model.

MORE FROM THIS AUTHOR: Vultr adds CDN to its cloud computing platform

Mature AI involves more than just operational and business efficiencies. It must also include responsible end-to-end AI practices. The ethics of AI underpin public trust. As with any new technological innovation, improper management of privacy controls, data, or biases can harm adoption (user and business growth) and generate increased governmental scrutiny.

The EU AI Act, passed in March 2024, is the most notable legislation to date to govern the commercial use of AI. It’s likely only the start of new regulations to address short and long-term risks. Staying ahead of regulatory requirements is not only essential to remain in compliance; business dealings for those who fall out of compliance may be impacted around the globe. As part of the right platform engineering strategy, responsible AI can identify and mitigate risks through:

• Automating workflow checks to look for bias and ethical AI practices

• Creating a responsible AI “red” team to test and validate models

• Deploying observability tooling and infrastructure to provide real-time monitoring

Platform engineering also future-proofs enterprise AI operations

As AI growth and the resulting demands on enterprise resources compound, IT leaders must align their global IT architecture with an operating model designed to accommodate distributed AI at scale. Doing so is the only way to prepare data science and AIOps teams for success.

Purpose-built platform engineering solutions enable IT teams to meet business needs and operational requirements while providing companies with a strategic advantage. These solutions also help organizations scale their operations and governance, ensuring compliance and alignment with responsible AI practices.

There is no better approach to scaling AI operations. It’s never too early (or late) to build platform engineering solutions to pave your company’s path to AI maturity.

You may also like…

Platform Engineering is not (just) about infrastructure!

The real problems IT still needs to tackle for platforms

The post Working toward AIOps maturity? It’s never too early (or late) for platform engineering appeared first on SD Times.

Platform engineering, and Instance Management Platforms, emerged as a way to codify and standardize the management of the platform including its CI/CD production pipelines. However, in the age of low-code/no-code (LCNC) platforms like the ones named above, applying platform engineering principles to these platforms is beneficial for non-developers and classic developers alike. LCNC platforms allow developers to immediately focus directly on developing sound business logic without coding the requisite application logic. Theoretically, this should shorten the time to market and lower maintenance costs since the platform handles all the application infrastructure (memory, storage, network, etc.). However, it’s critical not to overlook that organizations onboarding citizen developers will face the same challenges pro-coders see in enterprise development. 

Addressing the Root Causes of Chronic Delays

Most prominent players are still experiencing chronic delays in their operations, so they have turned to platforms. However, they often quickly find that even with these platforms, they are still experiencing chronic delays at pivotal times in the development lifecycle, which can be due to several factors. 

Inefficient deployment practices, slow approval processes, and lengthy manual testing all contribute to delays. Fixed release schedules are another big contributor. When companies can’t release on demand, they have to wait for the next change window, which limits how often they can release to production.

Beyond this, for companies using platforms like ServiceNow or Salesforce, processes like cloning databases or instances to serve as production environments can also be time-consuming. Cloning is typically used to copy production data/information to pre-production environments to test developed changes. 

While cloning is necessary to align production updates across all non-prod environments, this process (typically being database-heavy) can take up to 10, 20, or even 30 hours. That’s a lot of time for developers to sit idle; lost time is only the tip of the iceberg. 

These are just a few of the hurdles platform engineering teams are helping companies overcome, and they are doing it in a variety of ways. 

First, platform engineering teams and technology are helping to navigate the transition from fixed release schedules to on-demand releases by introducing better infrastructure, tools and processes that enable continuous integration and continuous delivery (CI/CD) pipelines. Beyond that, with automated deployment processes, companies can push changes to production without manual intervention, allowing for frequent and smaller releases.

Second, when it comes to processes like cloning, automation and accuracy are everything. If platform engineering teams can automate and accelerate their cloning process, they can minimize the discrepancies between source and target. The key is to establish and standardize better ways to minimize downtime and errors so that the platforms themselves can support a better service delivery standard. 

Who Owns that Delivery Pipeline?

Governance and standardization are crucial elements in the context of platform engineering. The platform engineering movement began when software engineers realized that building a CI/CD delivery pipeline involved significant coding. They recognized that the pipeline itself should be treated as an application platform, requiring a dedicated team of engineers. 

Many enterprises don’t anticipate hiring people specifically to maintain and build delivery pipelines. They might assume that using cloud services means everything is automatically taken care of. Consequently, part of the development team’s time is often allocated to managing the delivery pipeline as an application, which can be feasible since they are already responsible for app maintenance. This hidden burden is typically integrated into the overall maintenance costs of all the applications the development team is working on.

However, issues can arise in delivery pipeline governance when admin privileges become too widespread, and deployment practices too inconsistent. Beyond this, platform environments can spiral out of governance when there are too many changes in non-production environments. 

This is where we are seeing platform engineering teams begin to own the delivery pipeline, and introduce more automation surrounding governance and deployment flows and around the software development lifecycle in general. The reality is that platform teams should be looking to operationalize governance in the same way they standardize how code is developed, built, and deployed. The tools are out there to mindfully and intentionally embed governance in processes, and the results are helping teams to become better aligned. 

Keeping Environments as Production-Like as Possible

Often, when companies think about platform engineering, they think about the pipeline, not what environment the pipeline is passing through, or how to keep non-prod environments as production-like as possible. Without this alignment, the classic ‘works in development, not in production’ conundrum may be inevitable. 

Successful platform engineering teams keep environments as production-like as possible because they understand the value of testing and pushing tiny snippets of code to reduce the risk of something going wrong. When new functionality is tested in production-like environments all the way through, companies can demonstrably reduce the risk by size and volume, and improve quality. This is all part of the practice of scaling and building sustainable large enterprise systems

Ultimately, platform engineering has been tasked with solving the enterprise development problems encroaching on developer’s lives, and there is still a lot of work to be done. Without a strategic approach to managing platform engineering within modern LCNC platforms themselves, the enterprise development community won’t be anywhere near close to delivering at the speed today’s business demands without compromising quality or compliance.

You may also like…

Platform Engineering is not (just) about infrastructure!

Analyst View: What’s new, what’s now, and what’s next in platform engineering

The post The real problems IT still needs to tackle for platforms appeared first on SD Times.

It’s a relatively new concept, so in order to learn more about it, attend PlatformCon 2024, a week-long event that will bring together the most influential minds in the platform and DevOps space. Attendees can delve into various aspects of platform engineering from how to convince your manager, to how to actually build the perfect platform. 

Prepare to listen to Platform veterans and thought leaders like Kelsey Hightower, Gregor Hohpe, Charity Majors, Manuel Pais, Nicki Watt, Brian Finster, Mallory Haigh and many others! The event includes over 100 talks in total. 

15-minute lightning talks are divided into different tracks, including Stories, Tech, Blueprints, Culture, and Impact, allowing participants to explore the different dimensions of platform engineering.

PlatformCon 2024 will also feature a day focused on open source, where experts will share thoughts, discussions and highlights from the open source community.

The event runs from June 10th until June 14th and features top topics and speakers. Throughout the week, attendees can engage with speakers and fellow participants, gaining valuable knowledge and networking opportunities. The event concludes with a live closing event on Friday.

Attendees will be able to watch all talks at their own pace and join live virtual kickoff events for their region. Speakers will be available for Q&A on the Platform Engineering Slack.

Join platformengineering.org, the largest community of platform engineers out there and the lead organizer of PlatformCon. Learn from leading DevOps experts and connect with fellow platform nerds.

Register for free to secure your place and get updates on new speakers, opportunities to get involved, and in-person and virtual meetups.

Content provided by PlatformCon and SD Times.

The post PlatformCon 2024: The #1 Platform engineering virtual conference appeared first on SD Times.

What is new is the solution: platform engineering. Operating platforms sit between the end user and the backing services on which they rely. The platform is an internal software product, built by a dedicated team, that provides a curated collection of reusable components, tools, services, and knowledge, packaged for easy consumption. As illustrated below, the platform becomes a layer of abstraction between the developer and the messy complexities of operations.

The specific components and capabilities of each platform vary widely. Ultimately, the platform is whatever a development team needs it to be. The platform team’s job is to build that product. Each platform may be unique, but all platforms are:

• Productized: The platform is a product. User feedback directs product strategy.
• User-centric: Platforms solve users’ problems, not operators’. For example, developers probably don’t want a fast way to build a VM; they don’t want to think about infrastructure at all.
• Self-service: Developers can access everything they need from a single source, without opening tickets, sending emails, or filing requests.
• Consistent and compliant: Standards are built in, so that users cannot deliver code that is out-of-spec or insecure.

The platform is a “paved road” including both guidelines (recommended ways of travelling) and guardrails (hard boundaries the user can’t cross). A platform might enforce compliance guardrails: “You must run these automated tests of your security posture before deploying.” However, it might only suggest certain workflows: “We recommend the following tool for these use cases.”

What’s New: Platform Engineering Fulfills the Promises of DevOps

It’s important to distinguish platform engineering from what has come before. Automation is nothing new; nor are calls for better collaboration between developers and operators. A platform goes beyond existing techniques and tools. It is a new software product, with its own customers, lifecycle, user contracts, and lofty expectations.

Platform engineering represents the state of the art in DevOps. Not surprisingly, therefore, platform engineering has quickly become the hottest topic of conversation in that world, spawning its own user community and conferences. Gartner named platform engineering a Top Strategic Technology Trend in both 2023 and 2024 and predicts that, by 2026, 80% of large software engineering organizations will establish platform engineering teams.

What’s Now: The Platform Revolution Has Begun

IT shops have already begun to implement platform teams. Most start by building an internal developer portal, often using the open-source project Backstage. The portal is a central service catalog and document repository. It can also be a graphical user interface for automations and delivery pipelines. The user experience should be markedly better than whatever homebrew solutions developers have built for themselves. The goal is not to force developers onboard; they should want to use it.

Platforms make developers more productive. They free developers from the burden of building out their own operating environments. This allows developers to avoid unnecessary “glue” work and focus on writing code that creates value. When measuring the value of the platform—or making a business case to build one—focus on its positive impact on productivity. Deployment rates should increase; error rates, incidents, exceptions, rework, and time-to-value should all decrease. 

What’s Next: Platforms Expand and Evangelize

Platforms start small, often with only documentation and a service catalog. But even this is valuable. It saves developers from having to open tickets or send emails. Over time, the platform grows more capable. In the maximal vision of platform engineering, the platform has its own set of APIs. Developers write to the platform, and its abstractions become load-bearing parts of the application.

The platform can also expand to other users—data scientists, for example, or even business units looking to automate their work. They too will find value in a platform that meets users where they are. A platform team that can provide building blocks that are immediately useful, at an appropriate cognitive load, without unduly constraining users or forcing them into foreign ways of working, provides value now and in the future.

The post Analyst View: What’s new, what’s now, and what’s next in platform engineering appeared first on SD Times.

This limited view, however, does not give proper justice to the inherent richness and complexity of this field, a universe in which technology, innovation, and human engineering converge. Platform engineering extends far beyond the foundations of infrastructure, embracing a broad spectrum of technologies, practices, and philosophies that define the modern landscape of software development and systems architecture. Cloud infrastructure management, in particular, is only one piece of the puzzle.

In this article, we will briefly explain the relationship between infrastructure and platform engineering, and then focus on the other fundamental pillars that are not always immediately considered.

In this digital age, infrastructures have become more agile, scalable, and distributed, evolving toward increasingly sophisticated computing models such as cloud computing, multi-cloud, and hybrid cloud. Without a doubt, infrastructure management is one of the primary aspects to be considered (and probably one of the most important ones). Tools such as Kubernetes and infrastructure as code (IaC) tools have revolutionized the flexibility with which platform Engineers can build and manage these infrastructures, freeing them from traditional constraints and allowing them to focus on innovation and value addition.

But what distinguishes a platform engineering environment is not only the architecture on which it is based (or which it allows to be managed), but how this architecture is used to empower and simplify the work of developers and ops to make their work not only more efficient but also more rewarding and creative…

Read the full article, which originally appeared on ITOps Times, here, and to learn more about Kubernetes and the cloud native ecosystem, attend KubeCon + CloudNativeCon Europe in Paris from March 19-22.

The post Platform Engineering is not (just) about infrastructure! appeared first on SD Times.

Further, the introduction of developer observability into code, the use of value streams to eliminate bottlenecks and gain efficiencies, and the development of AI code assistants all aim to achieve that same goal. 

Lots of approaches, but has there been much success? The idea of “shift left,” where testing, security and governance moved into the developer purview, actually created more burdens on developers, which actually slowed productivity. Any number of DevX tools came to market in 2023, but research showed that organizations were buckling under the weight of tool sprawl.

And developer platform engineering was seen by many as tying developers’ hands and locking them into a platform they might not prefer.

It seems, then, that the complexity of the problem of making developers more productive was equal to the complexity of actually creating the applications that drive today’s businesses.

But the effort wasn’t all for naught. On the contrary, many organizations were able to increase productivity through hiring strong leaders who understand the role of developers and how they like to work.

In interviews throughout the year, effective management was cited as one of the biggest factors in developer productivity. Chris Harrold, developer experience program director at simulation software company Ansys, told SD Times in an interview earlier this year that the number one hallmark of a high-functioning team is trust – trust that your team is pulling together in the same direction, and that each member has the others’ backs. “Uncertainty kills,” he said.

Also, developers want their work to have meaning, and they want to work on interesting projects. Sometimes, though, that is at odds with the goals of the organization. Good dev managers can help by spreading the less interesting but important work around the team. “Something that I tell all my developers is ‘Look, you’re not always going to work on the latest, greatest, most amazing things all the time. Sometimes you’re just gonna build a button for a website,’ ” Harrold said. Some companies, he said, allow certain hours during the week for developers to go off and work on open-source projects or other things that are interesting to them, as a way to keep them recharged and rejuvenated. “And then when they have to build that button for the website, they can say,’ Okay, well, I got my one hour fix of really interesting work. Now, let me do what I’m doing.’ “

Platform engineering

The concept of platform engineering became top of mind in 2023, as organizations worked to make it easier for developers to innovate without having to worry about creating the environments to build, test and deploy their applications.

Platform company Humanitec, which runs PlatformCon, this year produced volume 2 of its State of Platform Engineering report, which showed that internal developer platforms (IDFs) are being widely adopted. It included the first-ever platform engineering maturity model, best practices and reference architectures, and looked at AI and the future of platform engineering.

In an SD Times Analyst View piece in May, Jason English of analysis and advisory firm Intellyx explained, “The decision to create a platform is a commitment to help developers of varying skill levels abstract away the complexity of underlying cloud native architectures with interfaces and tools atop readily configured environments. A platform engineering approach must offer ease of use, elimination of toil, and reduced cognitive load for development teams—helping orgs attract and retain the best talent.”

Using metrics, and the McKinsey report

The widespread use of DORA metrics has created a kind of standard way to measure things like deployment frequency, change lead time, change failure rate and mean time to restore. Ori Keren, co-founder and CEO of engineering efficiency company LinearB, said those metrics are totally relevant to engineering, but there are misconceptions when those are all you look at.

Organizations, he said, need to look at the metrics that are important to the business as well. 

LinearB’s benchmark report this year added something called planning accuracy, which shows how much a company committed to was actually delivered. “If you can commit to something and hit your goals with 80% of the features, that’s elite,” he said. “Most companies are not in those areas.”

Connecting those DORA metrics to the business is critical to understanding if you’re being productive in the business sense. “I always like this analogy to a car and an engine, so the engine works perfectly fine. But you could be navigating this car in the wrong direction,” he explained. “So DORA metrics are the RPM, how the car is operating, but you still need to balance those with the business metrics to know that you’re moving in the right direction.”

In August, McKinsey issued a report titled, “Yes, you can measure developer productivity,” which spelled out metrics beyond DORA that attempt to align productivity, joy and business outcome. It was widely panned in the industry as being “naive” and “ignores the dynamics of high-performing software engineering teams,” according to an article written in response to the report by Gergely Orosz and Kent Beck on “The Pragmatic Engineer.”

Coding assistants

The year 2023 saw an explosion of generative AI solutions to assist developers in writing clean, secure code. Microsoft Copilot, delivered in February, and IBM’s watsonx, which launched in May, as well as a number of others emerged, but they came with a caveat. Since today’s applications are cobbled together largely through the use of third-party and open-source components, it’s important to safeguard the output against licensing violations or improper use of those components.

According to Chris Wright, CTO at Red Hat, the question of using open-source code to train an AI model needs to be addressed. Does the license approve that kind of use, or with open source, do the creators just want to opt out of allowing its use in models? And what about then having to turn your code back into the open-source community?

These questions, and more, will be explored more fully in the coming year.

The post Year in Review: Developer productivity appeared first on SD Times.

Platform engineering is a discipline that brings together several different roles and integrates siloed technology into a single platform.

The new platform centers the developer experience, minimizing cognitive loads and making DevOps processes invisible. It achieves this through blocks, automations, and “golden paths.” 

The platform is also open and extensible so that platform engineers can make use of other DevOps tools in the industry, including CloudBees’ Jenkins. “This flexibility to orchestrate any other tool enables organizations to protect the investments they have already made in tooling. Teams can continue to use their preferred technologies simply by plugging them into the platform,” CloudBees wrote in a press release. 

It uses a self-service model to enable developers to be more autonomous and not have to wait on others for automations, actions, or resources.

The platform also centers security and includes out-of-the-box workflow templates with security measures already built in. CloudBees abstracts away sensitive information out of the pipeline, such as passwords and tokens. 

It also includes automated DevSecOps capabilities, such as security checks of source code, binaries, cloud environments, data, and identity. These checks are made possible by utilizing the Open Policy Agent project. 

The new platform also comes with frameworks for security standards like FedRamp and SOC2. 

“Today we are announcing the most open and extensible platform on the market, architected for cloud scale and the problems developers and platform teams face today,” said Shawn Ahmed, chief product officer of CloudBees. “Over the past 24 months we have listened, crafted, iterated, and developed a solution based on thousands of unique points of feedback. Time and time again, customers highlight the challenges in going fast, staying secure and improving developer experience. The CloudBees platform is the culmination of our commitment to reshaping the DevSecOps landscape. Our new platform empowers developers, unifies teams, and accelerates innovation while offering unprecedented flexibility and choice.”

The post CloudBees has a new DevSecOps platform specifically for platform engineering appeared first on SD Times.

According to a whitepaper by Humanitec, just five years ago, platform engineering was not a thing people talked about.

In the last decade, the concept of DevOps was all people thought about, ever since Werner Vogels remarked “you build it, you run it” at an AWS launch in 2006. 

This shift to DevOps caused a notable shift left in roles, where developers are now responsible for more aspects of an application’s life cycle and delivery workflow, all while the industry moved to more complex microservice architectures and technologies like Kubernetes, GitOps, and Infrastructure as Code (IaC), the report added. 

“Platform engineering emerged in response to the increasing complexity of modern software architectures. Today, non-expert end users are often asked to operate an assembly of complicated arcane services,” said Paul Delory, vice president analyst at Gartner. “To help end users, and reduce friction for the valuable work they do, forward-thinking companies have begun to build operating platforms that sit between the end user and the backing services on which they rely.”

The more successful engineering organizations invested in building Internal Developer Platforms (IDP) resulting in better performance on all DORA metrics. The IDP is the sum of all tech and tools that a platform engineering team binds together to pave a golden path or paths, according to Humanitec. 

Gartner forecasts that by 2026, the majority of software engineering firms (80%) will have created platform teams to supply mutually accessible services, components, and tools for application delivery. This will ultimately address the key challenge of collaboration between software developers and operators.

Lack of structural discipline is causing many platform problems today

The default assumptions and behaviors around what came before platforms were basically very bureaucratic or consultative, but not in a good way, according to Charles Betz, vice president and research director at Forrester.

“These groups have always been saying, well, if you need a computer, a virtual machine, or some compute resources or a database, you really need to have a server engineer and database administrator there to make sure you don’t get yourself into trouble. We’re going to be intimately involved in your technical design,” Betz explained. “We have overloaded people and it’s going to take long and unpredictable amounts of time for us to give you the designs and the analysis you need. Then oh, by the way, no, you can’t have access to any of the resources you need until we approve. And this is what leads developers to really dislike classic, traditional IT.”

However, Betz said that he also sympathizes with the IT side of things because IT professionals have typically been overloaded. They are often subject to unreasonable demands, and there have been histories of people developing systems with horrendous architectures, and then insisting that the infrastructure group take production ownership. 

“The reality is that the long periods of delay that the classic infrastructure group imposed simply is unacceptable in the modern Agile world,” Betz said. 

He added that people love to point to companies such as Netflix to try to mimic their way of being Agile, but those are the companies that are highly disciplined from a platform engineering perspective. 

“Netflix did not have autonomous product teams at a low granular level choosing willy nilly what products to use. That’s one of the myths that we often have to deconstruct when we come to a client. People say, well, all the product teams insist that they’re autonomous, and they can do whatever they want. And they point to Agile and various things. I’m like, you know, that’s actually not how it happened,” Betz said. 

The companies that succeeded imposed architectural discipline although they might have not called it that. But the bottom line is that they didn’t tolerate a lot of unmanaged variability and sprawl and that’s how they managed to be successful, according to Betz.

Humanitec’s DevOps Benchmarking Study 2023 found that giving DevOps tasks to developers as a way of implementing self-service is often executed poorly in many organizations. An IDP can enable developers to have greater self-service capabilities with the ability to spin up environments, deploy, roll back, and make changes to the architecture without relying on Ops. 

It is important, however, to keep the relationship between Ops and developers close while maintaining a separation of concerns. Through this, both teams can work together while having distinct roles.

The Humanitec report indicated that successful teams manage their app configurations across the entire organization in a standardized way. They also handle app configurations and infrastructure dependencies in the same manner, knowing how to distinguish between environment-specific and environment-agnostic configurations. These teams are more efficient at creating new environments and are able to provide more self-service when it comes to deployments, provisioning infrastructure, and assigning the infrastructure. 

The path forward for platform engineering isn’t about changing the engineering

As a whole, platform engineering is now undergoing a lot of experimentation and there isn’t an agreed-upon best practice out there yet. However, the biggest challenges facing platform engineering don’t always deal with the “engineering” part at all, according to Forrester’s Betz.

“When I went to the DevOps Enterprise Summit last fall, I talked to as many people as I could who were having success in platform engineering, and the one thing they all had in common was they were figuring out some way to bring product management principles to platform engineering,” Betz said. 

Platforms are products, according to the authors of Team Topologies, which suggests that the principles of organization, self-correction, monitoring environment, and setting standards are applied to the team. 

The challenge is that platform teams are now being created from the existing, outdated IT and operations organizations. These organizations are best suited for engineering purposes but are not necessarily well-equipped for the other tasks of a platform team. 

The problems are not in feasibility, which still continues to require high levels of engineering excellence, but rather in adding value, viability, and usability, according to Betz’s article, “Platform Product Management Versus Platform Engineering.”

This platform model encourages seeking automation wherever possible and managing queues anywhere else, making the internal offerings have advantages in terms of access to capital, reduced transactional friction, and maintaining high-quality service. Organizations should also implement explicit service design thinking in which employee net promoter score (eNPS) is tracked and customer journeys are understood, according to the article.

“To create a platform that is useful, you need to understand what is it the actual people you’re creating this platform for want and make sure that you’re regularly talking to them, iterating with them like you would have with any product backlog, product development, and product in the world of application development,” said Daniel Betts, senior director research analyst at Gartner. “You’re treating this in a similar way, you’re having to create a platform as an agile product.” 

Most of the teams that the products are targeted at are software developers, application developers, infrastructure engineers, and someone who’s creating code, creating assets for the business. 

“These people want to be able to create software or applications to deploy to a platform. They don’t want to have to think about tools, technology, governance, change management, all of those things. They want them all to be sort of made available to them and they want to focus on writing code,” Betts explained.

Often, platform teams are composed of a software engineer or two, as they add expertise to the benefits of writing machine-controlled code, code reviews, automated processes, and reusable components. They also aid in teaching scripting and coding best practices. Additionally, SREs can be part of the platform teams, as it is key for a successful product, Gartner’s Betts added. 

On top of that, organizations are having to develop platform engineering talent internally because there is a skills gap and they can’t find someone in the market that will know exactly how their platform works, according to Forrester’s Betz. 

“You have to do things like take somebody who’s a good technical engineer, and you’ve got to pivot them a little bit into somewhat less technical concerns like developer experience, product management, and products need to be valuable, viable, usable, and feasible,” Betz concluded. “We need to see more platform product managers.” 

The post The challenges with platform engineering don’t have to do with engineering appeared first on SD Times.