The jsii compiler (and jsii-rosetta) to follow TypeScript versioning

• Original Author(s): @RomainMuller
• Tracking Issue: #374

This RFC proposes to change the versioning scheme of the jsii compiler and jsii-rosetta to stop conforming to semantic versioning, and instead use the major.minor version of the TypeScript compiler it is built on.

Other packages (such as jsii-pacmak, @jsii/spec, ...) are not affected by this proposal and will continue to conform to semantic versioning. In particular, code generated by jsii-pamcak will continue to use the 1.x release line of @jsii/kernel, @jsii/runtime, and the languages' runtime libraries, so as to remain completely backwards compatible.

Working Backwards

Release Notes entry

Starting with this release, all packages in the *jsii* toolchain that internally
use the [TypeScript] compiler (i.e: `jsii` and `jsii-rosetta`) will be released
on a new release line that uses the same `major.minor` number as the
[TypeScript] compiler they are built against. All other packages (such as
`@jsii/spec`, `jsii-pacmak`, etc...) will continue to be released against the
same `1.x` release line they are currently released in.

The [TypeScript] compiler does not conform to [semantic versioning][semver]),
and instead guarantees the absence of breaking changes within a given
`major.minor` release line. We recommend developers model their dependencies on
`jsii` and `jsii-rosetta` using a minor-pinned version range (e.g: `~4.9.0`) in
order to avoid new [TypeScript] language versions breaking their codebases at
undesirable times.

New features will be introduced only in the *latest* `major.minor` release line
of `jsii` and `jsii-rosetta` (typically corresponding to the current stable
release of the [TypeScript] compiler). Breaking changes may be introduced with
every new `major.minor` release line and will continue to be documented in the
CHANGELOG.

Users may need to modify their code when upgrading from one `major.minor` release
line to another (addressing [TypeScript] language evolutions and `jsii` feature
evolutions). While upgrading to the latest release line of `jsii` helps minimize
the effort required for these updates, the release strategy allows developers to
do so on their own schedule.

This change is made to allow developers to benefit from the latest and greatest
features of the [TypeScript] language without requiring the entire ecosystem to
make the switch at the same time.

BREAKING CHANGE: In order to allow developers to use the latest & greatest
features of TypeScript, the `jsii` compiler and `jsii-rosetta` are now made
in-line with those of the `typescript` compiler (e.g: `jsii@4.9.x` is built on
TypeScript 4.9.x). Since the TypeScript compiler does not follow semantic
versioning, we strongly recommend you upgrade your `devDependency` on `jsii` and
`jsii-rosetta` to use a tilde range (e.g: `~4.9.0`) to be able to control when
you migrate to future [TypeScript] language versions. New `jsii` and
`jsii-rosetta` features will only be introduced in the _latest_ release line,
while older release lines only receive bug fixes until they are declared
end-of-life.

README.md for the jsii compiler

In order to allow developers to access and leverage the latest and greatest of
*TypeScript* language features, `jsii` compiler releases follow the `typescript`
package releases. For example, `jsii` version `4.3.x` is built on top of version
`4.3.x` of the `typescript` compiler.

Going forward, new `jsii` compiler features will only be introduced in the
_latest_ available release line (corresponding to the latest TypeScript
version), while older release lines will only receive bug fixes until they reach
end-of-life.

> IMPORTANT: The `typescript` package does not follow semantic versioning.
> Minor releases of the `typescript` compiler almost always include syntax
> breaking changes together with new language features. Since `jsii` releases in
> line with `typescript` minor lines, `jsii` does not adhere to semantic
> versioning either.

The `jsii` release notes for the initial release on each new *major.minor* line
will include a link to the corresponding TypeScript release notes entry, and a
description of any `jsii` breaking changes that may have been introduced with
the release.

When setting up a `jsii` project, we recommend pinning the dependency on the
`jsii` compiler to the desired minor version line (which corresponds to the
`typescript` release line), using a `~` SemVer range:

```js
{
  // ...
  "devDependencies": {
    // ...
    "jsii": "~4.9.0",
    // ...
  },
  // ...
}
```

README.md for the jsii-rosetta package

In order to allow developers to access and leverage the latest and greatest of
*TypeScript* language features, `jsii-rosetta` releases follow the `typescript`
package releases. For example, `jsii-rosetta` version `4.9.x` is built on top of
version `4.9.x` of the `typescript` compiler.

Going forward, new `jsii-rosetta` features will only be introduced in the
_latest_ available release line (corresponding to the latest TypeScript
version), while older release lines will only receive bug fixes until they reach
end-of-life.

> IMPORTANT: The `typescript` package does not follow semantic versioning.
> Minor releases of the `typescript` compiler almost always include syntax
> breaking changes together with new language features. Since `jsii-rosetta`
> releases in line with `typescript` minor lines, `jsii-rosetta` does not adhere
> to semantic versioning either.

The `jsii-rosetta` release notes for the initial release on each new
*major.minor* line will include a link to the corresponding TypeScript release
notes entry, and a description of any `jsii` breaking changes that may have been
introduced with the release.

When using `jsii-rosetta` in a project, we recommend pinning the dependency on
`jsii-rosetta` to the desired minor version line (which corresponds to the
`typescript` release line), using a `~` SemVer range:

```js
{
  // ...
  "devDependencies": {
    // ...
    "jsii-rosetta": "~4.9.0",
    // ...
  },
  // ...
}
```

Public FAQ

This section should include answers to questions readers will likely ask about this release. Similar to the "working backwards", this section should be written in a language as if the feature is now released.

The template includes a some common questions, feel free to add any questions that might be relevant to this feature or omit questions that you feel are not applicable.

What are we launching today?

We are announcing a change in versioning strategy for the jsii compiler and jsii-rosetta. Starting today, new releases of the jsii and jsii-rosetta packages will use the same major.minor version as the TypeScript compiler they are built on. Other packages in the jsii toolchain (jsii-pacmak, @jsii/spec, ...) are unaffected by this change.

Since the TypeScript compiler does not conform to SemVer, future releases of the jsii compiler will not conform to SemVer either. In line with the TypeScript compiler versioning scheme, breaking changes may be introduced in any new release that updates the major or minor version, but not when only the patch level changed.

This change enables jsii users to benefit from the latest and greatest features introduced in the TypeScript language as well as from performance improvements and bug fixes introduced in recent versions of TypeScript, without requiring the entire ecosystem to update compiler versions at the same time.

Why should I use this feature?

The new jsii versioning strategy gives developers more control over the TypeScript language version they are developing, without forcing an update schedule on them.

Developers are free to decide when it is appropriate for them to upgrade their jsii dependency to a new major.minor version line, bringing in new TypeScript language features as well as performance improvements and bug fixes.

Do I need to use the same jsii version as my dependencies?

The jsii compiler makes the due dilligence to ensure the artifacts produced (in particular, the .d.ts declarations files and the .jsii assemblies) remain compatible with previous versions, so that consumers of libraries need not update at the same time as their dependencies.

What happens to jsii v1?

Version 1.0.0 of the jsii compiler was released over two years ago (in February 2020), and will transition to the Maintenance tier of the AWS SDKs and Tools maintenance policy 6 months after this release.

During this initial 6 months period, jsii will continue to receive full support, including bug fixes and feature support, as it has in the past two years.

Once it enters the Maintenance tier, it will only receive fixes for critical bugs and security issues. The v1 release line will remain in Maintenace for 12 months, after which it will transition into the End-of-Life tier.

What is the support policy for these new releases?

Starting with release 4.9.0, new features will only be added to the current release line (the latest major.minor stream), which corresponds to the currently active TypeScript release line.

Critical bug fixes and security patches will be provided for previous release lines for 12 months, which covers 3 or 4 previous lines. Features will not be back-ported to non-current release lines.

Previous release lines will be considered end-of-life 12 months after they were superceded by a new current line.

Below is a chart of what this would have looked like if this proposal was enacted ahead of TypeScript 4.0 being released, and new jsii release lines were perfectly synchronized with TypeScript releases:

gantt
  title Support Policy Example (assuming 1 year maintenance)
  axisFormat %Y-%m
  todayMarker off

  section 1.x
  Current                 :crit, 2020-01-01,2020-08-20
  Active (6 months)       :2020-08-20,2021-02-20
  Maintenance (1 year)    :2021-02-20,2022-02-20
  End-of-Life             :milestone, 0d

  section 4.0.x
  Current                 :crit, 2020-08-20, 2020-11-19
  Maintenance (1 year)    :2021-11-19
  End-of-Life             :milestone, 0d

  section 4.1.x
  Current                 :crit, 2020-11-19, 2021-02-23
  Maintenance (1 year)    :2022-02-23
  End-of-Life             :milestone, 0d

  section 4.2.x
  Current                 :crit, 2021-02-23, 2021-05-26
  Maintenance (1 year)    :2022-05-26
  End-of-Life             :milestone, 0d

  section 4.3.x
  Current                 :crit, 2021-05-26, 2021-08-26
  Maintenance (1 year)    :2022-08-26
  End-of-Life             :milestone, 0d

  section 4.4.x
  Current                 :crit, 2021-08-26, 2021-11-17
  Maintenance (1 year)    :2022-11-17
  End-of-Life             :milestone, 0d

  section 4.5.x
  Current                 :crit, 2021-11-17, 2022-02-28
  Maintenance (1 year)    :2023-02-28
  End-of-Life             :milestone, 0d

  section 4.6.x
  Current                 :crit, 2022-02-28, 2022-05-24
  Maintenance (1 year)    :2023-05-24
  End-of-Life             :milestone, 0d

  section 4.7.x
  Current                 :crit, 2022-05-24, 2022-08-25
  Maintenance (1 year)    :2023-08-25
  End-of-Life             :milestone, 0d

  section 4.8.x
  Current                 :crit, 2022-08-25, 12w
  %% 4.8 is the current TypeScript release at time of writing...

gantt
  title Support Policy Example (assuming 6 months maintenance)
  axisFormat %Y-%m
  todayMarker off

  section 1.x
  Current                 :crit, 2020-01-01,2020-08-20
  Active (6 months)       :2020-08-20,2021-02-20
  Maintenance (6 months)  :2021-02-20,2021-08-20
  End-of-Life             :milestone, 0d

  section 4.0.x
  Current                 :crit, 2020-08-20, 2020-11-19
  Maintenance (6 months)  :2021-05-19
  End-of-Life             :milestone, 0d

  section 4.1.x
  Current                 :crit, 2020-11-19, 2021-02-23
  Maintenance (6 months)  :2021-08-23
  End-of-Life             :milestone, 0d

  section 4.2.x
  Current                 :crit, 2021-02-23, 2021-05-26
  Maintenance (6 months)  :2021-11-26
  End-of-Life             :milestone, 0d

  section 4.3.x
  Current                 :crit, 2021-05-26, 2021-08-26
  Maintenance (6 months)  :2022-02-26
  End-of-Life             :milestone, 0d

  section 4.4.x
  Current                 :crit, 2021-08-26, 2021-11-17
  Maintenance (6 months)  :2022-05-17
  End-of-Life             :milestone, 0d

  section 4.5.x
  Current                 :crit, 2021-11-17, 2022-02-28
  Maintenance (6 months)  :2022-08-28
  End-of-Life             :milestone, 0d

  section 4.6.x
  Current                 :crit, 2022-02-28, 2022-05-24
  Maintenance (6 months)  :2022-11-24
  End-of-Life             :milestone, 0d

  section 4.7.x
  Current                 :crit, 2022-05-24, 2022-08-25
  Maintenance (6 months)  :2023-02-25
  End-of-Life             :milestone, 0d

  section 4.8.x
  Current                 :crit, 2022-08-25, 12w
  %% 4.8 is the current TypeScript release at time of writing...

gantt
  title Support Policy Example (current + previous)
  axisFormat %Y-%m
  todayMarker off

  section 1.x
  Current                 :crit, 2020-01-01,2020-08-20
  Active (6 months)       :2020-08-20,2021-02-20
  Maintenance (1 year)    :2021-02-20,2022-02-20
  End-of-Life             :milestone, 0d

  section 4.0.x
  Current                 :crit, 2020-08-20, 2020-11-19
  Maintenance             :2021-02-23
  End-of-Life             :milestone, 0d

  section 4.1.x
  Current                 :crit, 2020-11-19, 2021-02-23
  Maintenance             :2021-05-26
  End-of-Life             :milestone, 0d

  section 4.2.x
  Current                 :crit, 2021-02-23, 2021-05-26
  Maintenance             :2021-08-26
  End-of-Life             :milestone, 0d

  section 4.3.x
  Current                 :crit, 2021-05-26, 2021-08-26
  Maintenance             :2021-11-17
  End-of-Life             :milestone, 0d

  section 4.4.x
  Current                 :crit, 2021-08-26, 2021-11-17
  Maintenance             :2022-02-28
  End-of-Life             :milestone, 0d

  section 4.5.x
  Current                 :crit, 2021-11-17, 2022-02-28
  Maintenance             :2022-05-24
  End-of-Life             :milestone, 0d

  section 4.6.x
  Current                 :crit, 2022-02-28, 2022-05-24
  Maintenance             :2022-08-25
  End-of-Life             :milestone, 0d

  section 4.7.x
  Current                 :crit, 2022-05-24, 2022-08-25
  Maintenance             :2022-08-25, 12w

  section 4.8.x
  Current                 :crit, 2022-08-25, 12w
  %% 4.8 is the current TypeScript release at time of writing...

Internal FAQ

How will we maintain backwards compatibility?

The change only affects the TypeScript compiler version used internally by the jsii compiler and the versioning scheme for the jsii package itself.

The compiler will continue to emit .jsii assemblies that conform to the schema defined in the @jsii/spec package, which will hence continue to be compatible with all other tools part of the jsii toolchain (including jsii-pacmak, ...).

In order to maximize compatibility between TypeScript compiler versions, and since TypeScript occasionally introduces backwards-incompatible syntax changes (additions, modifications) to the declarations files (.d.ts), the jsii compiler will proactively produce down-leveled declarations files targeting TypeScript compiler versions used by previous (not yet end-of-life) releases of jsii. This can be achieved using the downlevel-dts utility. Failure to do so may make it impossible to use a library compiled with a given release of jsii in a project that is still using an older release, which would be akin to forcing dependents to upgrade at the same pace as their dependencies (which is undesirable).

Why are we doing this?

Before this change, jsii users had been stuck with TypeScript 3.9 for a very long time, due to the introduction of several breaking changes in the language specification: upgrading the TypeScript compiler that jsii builds on would cause existing code to break.

In order to allow each package author to decide for themselves without hinging on their dependencies' choices, or influencing their dependents, we are making it possible for every developer to decide when they want to migrate to a new version of the TypeScript language, by following TypeScript's versioning for the compiler, and continuing to perform coordinated releases of every other package in the toolchain (those are unaffected by this change).

The jsii-rosetta package also internally uses the TypeScript compiler that needs to be compatible with the TypeScript language level used by the project. Developers hence need to also be able to control which TypeScript version is being used by jsii-rosetta if they want to use this.

Using the same major.minor version as the TypeScript compiler makes it easy for developers to identify which TypeScript language features are available to them or not. As the TypeScript compiler does not conform to semantic versioning, this implies jsii and jsii-rosetta also need to stop conforming to it.

Why should we not do this?

Semantic versioning is the dominant versioning scheme in the JavaScript world, and diverging from it might break assumptions customers make when consuming packages from npm.

Following TypeScript versions means releasing new major.minor releases (that may include breaking changes) more often than is currently done (the TypeScript maintainers typically declare a new major.minor line four times per year: in February, May, August and November respectively). This is a lot more release lines than other AWS products offer, and it would not be reasonable to uphold the AWS SDKs and Tools maintenance policy, as it requires offering full support (features and bug fixes) for "Generally Available" releases for a minimum of 24 months (this would require providing full support for 8 different releases, and back-porting automation is likely to be difficult due to the amount of breaking changes in the TypeScript compiler API between releases). Diverging from this policy is a significant change that customers may not expect.

Additionally, since we cannot afford to back-port new features (especially if significant) on older releases, this means users may need to update their code in order to gain access to new features introduced in the compiler. While this may be a desirable forcing function on developers to upgrade to the latest release line, it may impose challenges when such features cannot be implemented in a way that is compatible with previous versions of the compiler (e.g: it may be difficult or impossible to implement RFC 193 in a backwards compatible manner, as it likely requires the introduction of a new type kind, which previous release lines would not recognize).

What is the technical solution (design) of this feature?

The proposed delivery plan for this feature is as follows:

• Communicate about the upcoming change in new releases on the v1 release line, including a planned timeline for the initial release of the v4.9 line, and language about the departure from SemVer.

• Optional: Separate the jsii compiler from the rest of packages in the jsii toolchain into separate repositories. This would make it easier to release the jsii compiler separately from other parts of the toolchain that follow a different versioning scheme.

  • Move other packages to a new mono-repository (or to several new individual package repositories), e.g: github.com/aws/jsii-toolchain.

  • Make the github.com/aws/jsii repository by a single-package repository.

• Update the release automation to use the TypeScript compiler version's major.minor level, and only update the patch level on new releases.

• Upgrade the TypeScript compiler internally used by jsii to the current latest release of the typescript package.

• Make the jsii compiler transparently prepare down-leveled declarations files to ensure backwards-compatibility of compiler outputs with previous releases of jsii. This can be done using the downlevel-dts package to produce declarations files compatible with TypeScript 3.9, and adding a typesVersions key to the package.json file of packages.

  • See proof-of-concept: aws/jsii#3501

• Release the initial jsii release on the 4.9 line (for practical reasons, we will disregard TypeScript releases between 3.9 and the current version when this proposal is implemented, which at time of writing is 4.8).

• Formally announce that the v1 release line of jsii will move into the Maintenance tier of the AWS SDKs and Tools maintenance policy in 6 months, and explain the policy that is applicable to newer releases.

• 6 months later: Formally announce that the v1 release line of jsii is entering the Maintenance tier of the AWS SDKs and Tools maintenance policy, and will continue to receive critical bug and security fixes for 12 months before transitioning to end-of-life, and repeat the maintenance policy that new releases will benefit from.

• 12 months later: Formally announce that the v1 release line of jsii is transitioning to end-of-life and will no longer receive any updates. Repeat the maintenance policy that is applicable to newer releases.

Is this a breaking change?

This is not a breaking change in the traditional sense, however this is a significant change in versioning schemes used for the jsii compiler, and a departure from the AWS SDKs and Tools maintenance policy.

As such, this warrants ample communication with our customer base to avoid they are caught by surprise, or have broken expectations.

What alternative solutions did you consider?

Bring-your-own TypeScript version

One of the main rationales for this change is to enable the use of new TypeScript releases without forcing the entire package ecosystem to make the change at the same time (due to language-breaking changes).

An option to address this would be to allow customers to "bring their own" TypeScript compiler, by making it a peerDependency. This however comes with a significant challenge: the TypeScript compiler API is not stable between TypeScript releases. Addressing this would require maintaining an adapter layer for each supported version of the TypeScript compiler, and this would require tremendous efforts, in particular as there appears to be no way in TypeScript to re-export a namespace including all the type declarations it contains, as typeof ts (assuming ts is the TypeScript compiler namespace) only represents entities with a run-time value, and none of the interfaces it contains. The consequence of this is that writing an adapter for a particular TypeScript version requires wrapping all useful functions of the compiler API and re-declaring the types they use, which represents hundreds if not thousands of lines of code for each supported version.

Only de-couple the jsii package version, but stick with SemVer

It would technically be possible to de-couple the jsii package from the rest of the toolchain's versioning, and to issue a new jsii major version each time we update the TypeScript compiler it internally uses. This would however make it difficult for customers to understand the relationship between jsii major releases and the TypeScript language level they support.

Using the same version number prefix (major.minor) as the TypeScript compiler makes the relationship clear and removes the need to maintain separate documentation for customers to understand what they are upgrading to.

Release a different package for each supported TypeScript version

The release model used by cdk8s-plus involves releasing a different package for each version of the underlying resources (e.g: cdk8s-plus-22 targets version 22 of the resources). A similar model could be used for jsii, where a package named jsii-4.9 could be released for the compiler that builds on TypeScript 4.9.x.

This release pattern however creates an opportunity for supply-chain attacks as a malicious party could release jsii-4.10 ahead of us and possibly exploit unsuspecting users.

The attack vector can be mitigated by scoping the packages since npm scopes provide ownership guarantees, which would require releasing @jsii/typescript-4.9. However this approach makes it more difficult for customers to know when a new release of the compiler is available, since dependency maintenance automation will only look for new versions of the same package.

What are the drawbacks of this solution?

This solution is a stark departure from versioning schemes and maintenance policies used by AWS on open-source products. It implies offering critical bug fixes and security updates for up to 5 different release lines (1 current, and 3 to 4 previous), which might require significant effort.

Following the TypeScript major.minor reduces our ability to introduce breaking changes to the jsii compiler, as these must be timed together with TypeScript major.minor release timelines. However, since TypeScript releases a new major.minor line 4 times a year, this is not a blocker, but will require careful planning.

Breaking changes that could be tempting to introduce in new major.minor release lines include:

• Upgrading default severity of some diagnostic messages to ERROR
• Adding new compile-time validations
• Dropping support for a recently end-of-life node version (this happens once or twice a year)
• Dropping support for a particular TypeScript language version (for example, DefinitelyTyped packages only support TypeScript compilers that are less than 2 years old)
• Changing configuration file syntax, format, default values or required entries
• Addressing a security issue that requires breaking existing code

What is the high-level project plan?

Are there any open issues that need to be addressed later?