Along with active participation, it is very important to look after the ROI of open-source projects, programs, and contributions. The open-source program managers need to take the initiative of demonstrating the ROI of their efforts and endeavors.

This article today is all about understanding what exactly is an open source program, and what are the convenient ways for measuring open-source projects’ success. Let’s get started. 

Things to know about an open source program

This section introduces you to some of the most important information about an open source program.

What do you understand about an open source program?

A central open source program office can be considered a designated place where basically open source can be well supported, nurtured, explained, shared, and developed inside an organization. 

With such availability of an office in place, businesses can look forward to establishing and implementing their effective open source strategies in a very clear way, providing their leaders, marketers, developers, and other staff members with the important tools they require to make open source a huge success within their operations.

Need for an open source program

Open source software is broadly used by companies today in a much wide range of industries such as retail, finance, automotive, and many more, but sadly it isn’t well understood by the executive leaders and decision-makers who happen to run the operations.

One of the major concerns is that when organizations create and follow traditional business plans which tend to dictate their operations and goals, open source software can be confusing. You’ll find that open source innovation has its own methodology and doesn’t really follow traditional business procedures. 

There is even one big difference which is open source development is collaborative and on the other hand, the traditional software and business practices are actually proprietary and closed. 

Also, for a lot of businesses, the required change in philosophy while approaching open source use doesn’t really come easy and naturally.

That’s when exactly the creation of an open source program can be considered a benefit. So, if a company happens to create an open source program office, businesses can easily organize and streamline the use of an open source in such a way that it directly gets tied up to a company’s long-term business plans. 

In fact, an open source program office is considered the center of the universe for a company’s open source operations and structure, as it enables bringing all the required components together. 

An open source program office can comprise of setting code use, selection, distribution, auditing, and even other policies, and also training developers, providing legal compliance, and promoting and building community engagement as well. 

Role of an open source program

If your organization successfully creates an open source program office then, it will encourage standard coding and organizational practices, procedures, and toolsets enabling your organization to reach aspired goals and objectives.

Along with that, a program office helps in removing or avoiding unnecessary, rigid procedures that the creative developers might avoid, which can be proved as a threat to security and other aspects of projects.

The responsibilities of a program office can be varied. These comprise:

• To clearly communicate the open source strategy within and outside the organization
• To own and oversee the implementation of the strategy
• To facilitate the effective use of the open source in commercial products and services
• To ensure high-quality and frequent releases of code to open source communities
• To engage with developer communities and see whether the company contributes back to other projects effectively or not
• To foster an open source culture within an organization
• To maintain open source license compliance reviews and oversight

So, for each and every company, the role of the open source program office tends to likely be custom-configured, depending on its business, goals, and products. 

You won’t find any broad template in order to build an open-source program that can be applied across all industries or let’s say across all companies in a single industry. 

Therefore, the creation of an open source program office can be challenging, but you can obviously learn from the other companies’ experiences and utilize such approaches to effectively fit your own company’s requirements. 

Example of an open source program

Let me here give you an example of a company handling its own open source program office. So, Microsoft has been seen working over the past couple of years to create and also refine its own open source approach. 

You’ll find that thousands of employees in a broad range of business units are putting immense effort to make their open source program office a huge success. 

This company has given a huge priority to this program office for assisting its developers, marketing teams, and others who are engaging with open source for hardware and software products, cloud services, content, media, games, and other product lines.

So, it has been proved that every division needs different assistance depending on its individual business models and engagement scenarios. And, Microsoft is putting its best efforts into utilizing the maximum benefits of the open source program office.

Key metrics to track for measuring open source program’s success

This section talks about the top metrics that can be used for assessing the overall project health in your open-source program. Such tracking can be regarded as a starting point for a much more thoughtful and rigorous analysis. Let’s take a look at it. 

Number of contributors (and ratio of internal to external contributors)

Usually, projects can be seen getting started with the majority of contributions coming from internal developers and also from outside contributions. The healthiest projects tend to have extremely diverse communities with a lot of contributions coming from various other companies. 

So, the projects that consistently attract new external contributors succeed in doing a good job by properly maintaining the projects, welcoming new contributors, and incorporating feedback from the community.

Number of pull requests submitted, opened, and accepted (and time remaining open)

When a contributor identifies a bug or has a feature request that they can (and have clearance to) patch or even write by themselves, they certainly do so and submit it as a pull request (PR). 

So, tracking the number of pull requests, and what exactly happens with them, reveals the number of codes that are being submitted by contributors outside of your employment and is, therefore, an indicator of the level of outside contributors’ interest in your projects. 

Number of issues submitted (and length of time remaining open)

Users who do not have enough time, authority, or capacity to submit a pull request, but witness problems with your code can actually submit their bugs and feature requests as an issue. 

The number of issues, and the manner in which they are addressed, help in indicating your projects’ levels of user adoption and how responsive maintainers look after the user needs. 

And this number depends completely on how issues are being tracked. 

Number of commits per contributor (internal and external)

The efficiency of a project can also be ascertained by the number of external commits a project has in proportion to the total numbers. Healthy projects tend to observe the ratio of external contributors that increases over time. 

By measuring the number of commits per contributor, you can check whether your projects are able to attract new contributors and also if those contributors can be sustained for a long time. 

Number of external adopters

It is very important that every open-source project has a certain way to track companies that look forward to adopting the software in a production environment. It can either be through an ADOPTERS.md file or a simple list in the README, but the main point here is to track this list and make sure that it properly grows over time. 

If by any chance the number of external adopters stops growing, then it can indicate everything starting from project maturity to project obsolescence.

Number of projects created or contributed to (program-wide)

You can track these metrics for each project your organization releases, as well as the projects your developers are actively contributing to. While creating your open-source strategy, you need to be identifying the business-critical projects your organization is utilizing and earmarked some investment for particularly contributing to those projects. 

It’s essentially important to measure your organization’s open-source success not just by your project’s health but by its overall open-source activity at large.

Other important metrics to track

Depending on your specific goals, you can also track the success of your open-source projects with these important metrics. But do remember that achieving a big number isn’t your ultimate goal. 

Along with the results, the process of tracking and finding patterns in the data that can bring an overall improvement in the projects is equally important. Below are some more metrics that you can possibly track.

Popularity or awareness

You can start with the number of visitors to the project website, followed by the total number of followers on GitHub/ GitLab. Then comes the number of followers on social media accounts such as LinkedIn, Twitter, or Facebook.

News clips and media mentions are to be measured as well. Finally, you can keep a count on the number of meetups organized and hosted (e.g., via meetup.com). 

Program costs

You can keep an eye on the memberships and donations, infrastructure and support, conference attendance and travel, tools, training, and staff: engineering, PR & marketing, and legal. 

Influence

You can track the number of contributing external companies, number of employees in a maintainer/leadership role in your strategic projects, diversity of contributors to your projects, number of forks created, patches rejected, and why, number of downloads, and stages of adoption (# of deployments in PoC and production).

How to measure your success as an open source program manager?

Now, I would like to take you across how open-source program managers can identify whether they are on the right career track or not. And the below ways ascertain whether they are justifying their job roles and contributing to the open-source projects’ success.

The first thing an open-source project manager can look for is whether people think he is capable enough to answer their questions even outside his job scope. E.g., if anybody comes to you with any queries that they believe you know or at least can find ways to answer, you can take that as a compliment. 

A program manager can take the assurance that being approached to resolve any queries, he has done a great job of building credibility with the community. 

The next thing a program manager should look for is if the people have trust in them or not. Do they feel safe to be transparent regarding their work progress? E.g., if things are not going smoothly and the team is falling behind the deadlines, do they feel safe to inform you about it? 

If they don’t, then, unfortunately, you’re not being able to build a culture of trust among your people. So, as a program manager, you should be able to make your team feel free to talk about any concerns which can be well handled by you. 

By doing so you can take the satisfaction of making meaningful contributions to the open-source community.

When things go smoothly, it essentially doesn’t mean that the project release is on track, or all bugs are fixed and everything is simply perfect. So, running smoothly isn’t about the absence of bad things, but it’s about the absence of surprising bad things. 

As a program manager, if you’re able to maintain full work transparency in regards to any challenges or work issues and find a collective effort to resolve them. Then you can definitely consider yourself a successful program manager. 

Learn more here:

• How to ensure open source sustainability
• What are the merits of open source DXP? 
• How to lead open source the right way? 
• Guide on open source security 

Conclusion

From this article, it is very evident that organizations need to evaluate their open-source projects, programs, and contributions in the best way possible. Without measuring the overall success of the open-source project, we cannot identify the places where we need to grow and improve for better contributions in the coming years.