Abstract

Pekko is a toolkit and an ecosystem for building highly concurrent, distributed, reactive and resilient applications for Java and Scala.

Proposal

Pekko is a toolkit that brings the actor model (popularised by Erlang) to the JVM, providing the basis for building both locally and distributed concurrency. On top of this Pekko provides a rich set of libraries built on top of Actors to solve modern problems, including:

  • Streams: Fully bi-directional backpressured streams following the Reactive manifesto
  • HTTP: A fully streamed HTTP client/server built on top of streams that also provides expected tools (such as connection pooling) necessary for highly available web services
  • connectors: A rich set of connectors for various databases, messaging, persistent services built on top of streams
  • grpc: A gRPC server/client
  • projection: Provides abstractions necessary for CQRS pattern such as envelope, necessary for systems such as Kafka.

Background

Pekko is a fork of the Akka project just before its licence changed from Apache 2 to Business Source License 1.1. The project provides a set of tools and frameworks that covers the complex problem space of distributed concurrent systems. It is designed to support the design principles of the Reactive Manifesto by providing components to efficiently scale up systems within a server or scale out across multiple servers, are high performance, resilient to failure, distributed systems without a single point of failure.

Rationale

There is a large cohort of applications and libraries that were dependent upon the original open source version of this project. Numerous developers contributed their time in the belief that the project would stay open source. When the licence was changed the work of those developers was locked up and a vital resource for the cohort of applications and libraries disappeared. Apache Flink is an example of a library that used the original library upon which this project is based. This project is to continue the open source development that was promised under the original Apache 2 licence. We ask that the Apache Foundation accept this project so as to prevent any future incompatible licence switch in the future.

Apache has a long standing tradition of not accepting hostile forks. There has been some discussion of whether this project violates that tradition. We believe that it does not.

For many years, Lightbend has been a steward for this open source project, attracting contributions from many developers and building a community under the Apache License. It's within their rights to offer their future work under a different licence. The Pekko project will provide the continuity of an Apache-licensed home for long-term support, maintenance and new features for the developers that wish to continue using and building on their previous work. The major historical reason why Apache would be a good home for Pekko is that it will protect the project from licence changes similar to what is instigating the initial incubation proposal. If Pekko becomes part of Apache then it gives confidence to the community/users of Pekko that such an incident won’t happen in the future again. There are also currently existing Apache projects such as Flink that use Akka to varying degrees and hence having Pekko to be part of Apache gives confidence to these other Apache projects. We believe that this fork is a maintenance of the pre-existing Apache 2 licence and ask that the Apache community view it as such.

In general, Akka had a very large penetration in both Scala and Java codebases, particularly in large scale enterprise projects/systems. Since it is a JVM based library it fits well into the Apache ecosystem. We feel that we can contribute to the stability of existing Apache projects as Apache can contribute to the stability of Pekko.

Initial Goals

Due to the sheer size of Akka, initial goals will be largely be adjusting and modifying the codebase so its fit for community orientated contributions, this includes:

  • Removing the Akka trademark from all sections in the code.
  • Setting up the build system using github actions to make sure we have a CI system setup whenever pull requests are made (some Akka projects use github actions already, in which case we need to make sure it still works after the work).
  • Also using github actions for Maven/Github releases, in Scala/SBT projects it's typical to trigger a release when a person with necessary rights pushes a signed git tag. Also need to find a solution for the official Apache source SVN release.
  • Other adjustments such as using testcontainers to minimise the friction in running tests for projects such as containers.
  • Update all of the documentation to be appropriate for an Apache project (as well as using the project’s name).
  • Assure continuity of security fixes and update on the existing Apache-licensed code, which might otherwise be at risk.
  • Continuously building community and identifying the contributors that might assume project management roles for a successful graduation.
  • Port and/or merge currently open contributions/PR’s from Akka community, if they are willing, which are currently frozen due to the licence change.
    • This will be done after the first release of Pekko to accommodate for companies that were using Akka so that they can use a version of Pekko that is functionally equivalent to the last Apache release of Akka.

Current Status

Meritocracy:

The project upon which Pekko is based was initially very welcoming to contributions from external developers. However, they were controlled by a commercial entity. Some projects were labelled as “community driven projects”, those would not have the guarantee from the entity to have a dedicated person working on them, so they needed to rely on community participation. The entity allowed trusted external developers to commit to the code base and have write access to these mentioned community repositories. The new management team understands that, and hopes that developers will become engaged with the project and will help drive the development of processes as well as code so that we can become the welcoming development team we hope to be.

Many of the core developers are experienced Apache developers working on other projects. Through their understanding and the work of the mentors we are assured that a meritocracy will thrive around this community.

Community:

Akka being an already established open source project already has a community. This community was present in many channels, such as

  • Scala reddit (/r/scala)
  • Akka gitter channel
  • Lightbend forums
  • Scala community forums

It is expected that once Pekko is launched a portion of this community (specifically the ones interested in open source contributions) would migrate to Pekko’s mailing list.

Early discussions in existing community forums have shown that there is interest in an Apache project and we have commitment from several channels to contribute to this project.

Early participants in the discussions forming this project are participants in the above channels and have a broad reach that will bring other developers into the project. In addition to software developers we have documentation specialists and test engineers participating. We have a broad community. And while it does not have the depth we would like, we believe that it will continue to grow as we move forward.

Core Developers:

The individuals on the committer list are ones who have a history of contributing to the Akka project before the licence change as well as individuals that participated in community discussion on various Akka forums (i.e. Lightbend forums). In addition the individuals have a history of working and contributing to open source projects.

Sean Glover is a former member of the Akka team. He primarily maintained Akka Streams and related projects Alpakka and Alpakka Kafka. He currently uses Akka on the job and in other OSS projects he maintains.

Phillip Warburton feels he is not enough of an expert to contribute code but is interested in contributing in to document clean-room fixes.

Andy Chen is a contributor on Akka and his work is based on the Akka libraries.

Jean-Luc Deprez works on a small team, mid sized software company, subsidiary of a large corporate group. His team built a project on top of Akka. The September 7th licence change came as a pretty hard personal blow. He will divert his community participation this way, contributing in discussions and issues. If the team figure out a fix, you can count on a PR.

Greg Methvin is a maintainer on Play Framework (which is now community-maintained). We are evaluating whether this can serve as a replacement for Lightbend Akka going forward, as opposed to moving off Akka entirely.

Nicolas Vollmar is a maintainer on the akka-kryo-serialization project.

PJ Fanning maintains a few akka related projects: swagger-akka-http and micrometer-akka.

Daniel Schröter is a maintainer of the akka-kryo-serialization project. Altoo is using akka and has been regularly submitting bugfixes/improvements to akka.

Josep Prat is the top 5 contributor of the Akka HTTP module and did all contributions during his spare time, also he is in position 45 of all time committers in the Akka project.

Matthew de Detrich is a contributor to both Akka and Alpakka, submitting extra feature improvements as well as the wider Scala OS ecosystem in general (i.e. contributions to Scala stdlib itself, Scalatest etc etc). Most of these contributions were done in his spare time.

Alexandru Nedelcu created Monix which is one of the popular reactive/future/IO implementation used within the Scala ecosystem that adheres to the Reactive Streams protocol. He is a very prolific contributor to the Scala OS ecosystem, having been previously part of the Typelevel steering committee and also happens to works at a company that heavily uses Akka for its payment systems.

Johannes Rudolph is a major contributor to akka and the lead developer for akka-http.

Alignment:

Apache Flink is using Akka however they are looking for a replacement due to the licence change. There may be other Apache projects that can use Pekko.

We believe that we can be a good community member.

Known Risks

Project Name

Several names were floated to replace the Akka name. There was a community discussion and vote around the name

We have selected the name Pekko after a vote on candidate names. Akka is the Finnish goddess of fertility and the earth. Pekko is the FInnish god of farming & protector of the crops which create a nice analogy in a sense. We pronounce the name Peck-O.

A search of the WIPO database and EU IPO Search did not turn up any active Nice category 9 trademark registrations containing the phrase “pekko”. All results were “pekko” compounded with other phrases.

Orphaned products

Pekko provides libraries that are used by major companies, Apache projects, and individuals. The initial development team comprises representatives from these organisations. We believe that Pekko will attract open source developers that have contributed to Akka in the past and want to see their work continue to be open source. We are here, not because we are a potentially orphaned product, but rather because the contributors to the existing product want to see the open source project continue.

Inexperience with Open Source:

The project upon which Pekko is based was, as noted above, an open source project. The developers that wish to continue with the development understand open source projects. Several of the developers are committers on other Apache projects. Several are members of open source project offices at their respective employers. Several of our members are experienced in developing communities around open source projects.

Length of Incubation:

We expect that the incubation process will be quite long as we have significant code cleanup and documentation scrubbing to be completed. In addition we need to configure the Apache build systems to properly build what is a fairly complex project (i.e. akka core has tests that require multi node machines). An incubation of 1-2 years would not be unexpected.

Homogenous Developers:

The current list of committers spans Asia, Europe, and the Americas. All are experienced in working in distributed environments. While there are cultural differences, all are committed to open source development.

Reliance on Salaried Developers:

While several of the developers have the same employer, no employer has specifically stated that they are committing developers to Pekko. Any commercial contributions of time/effort are due to the need for changes/fixes to the libraries used by the commercial entities.

All developers have shown a specific interest in keeping this project open source and we expect that future developers will join for the same reasons.

Relationships with Other Apache Products:

As noted above Akka is used by Apache Flink though Flink is planning to migrate away. Pekko may still turn out to be useful for them. We expect that other projects may find the library of use.

A Excessive Fascination with the Apache Brand:

We understand the need to protect the Apache brand. We also understand that the Apache brand brings a licensing guarantee. While we are desirous of the licensing guarantee, we believe that the project we are bringing is viable and important. We think that it will provide support for at least two existing Apache projects and contribute to the strength of the Apache brand world wide.

Documentation

The current documentation can be found at https://akka.io/docs Other documentation is available as .md files within the source tree of the project.

Initial Source

The original source comes from the Lightbend Akka github repository before they transitioned to the Business Source License. The code base to be imported resides in several repositories under the control of Matthew Benedict de Detrich, the base one being https://github.com/mdedetrich/akka-apache-project

Source and Intellectual Property Submission Plan

Since Pekko is forked from an already existing Akka codebase there is a high chance that there may be existing IP/trademarks in the codebase that needs to be screened. The Akka name itself needs to be changed/removed entirely from the codebase.

In addition there are already existing github triggers and other mechanisms that build and test the system. We will need to review the triggers to determine how to implement the same or similar functionality on the Apache infrastructure.

Our high level plan is to:

  1. A lift and shift of the existing github core repository into the Apache repository.
  2. Modification of the build and test process to run on the Apache build infrastructure.
  3. Emphasis on the dev and user mailing lists for help requests and discussions.
  4. Development and modification of project documentation.
  5. Create a “release” 0.1.0.
  6. Verify release
    • Functionality
    • Documentation
    • Meets Apache standards
  7. Continue using the above process to migrate additional libraries to the Apache libraries with each one being a release “0.x.0”
  8. Upon completion of the migration of all pertinent libraries release 1.0.0

This strategy gives us the opportunity of learning the Apache way and the Apache infrastructure with small modules before we move into the more complex systems, building up layer by layer until we have a complete first release.

External Dependencies:

The initial code was licensed under Apache Source License v2. As such we believe that it meets all the requirements for external dependencies. However, as part of the incubation process we will verify that all dependencies have appropriate licences. Any dependencies that do not meet the requirements will be removed and alternative libraries or code used instead.

Cryptography:

We believe that there is no cryptographic code in the code base. However, as part of the incubation process we will verify that this is true.

Required Resources

Mailing lists:

Subversion Directory:

We do not intend to use subversion.

Git Repositories:

Issue Tracking:

  • JIRA Pekko (PEKKO)

Initial Committers

There has been a lot of interest in being an initial committer, and we've tried to pick a fair representation of interested developers from the requests. We want to be clear that we welcome all contributions and expect that the incubation period is the right time for this list to grow and evolve.

Sponsors

No company has committed to providing dedicated developers for the project but Aiven and Altoo have committed to supporting development as needed by their respective organisations.

Champion:

Nominated Mentors:

Sponsoring Entity:

  • The Incubator  
  • No labels