Making Java nullable fields backwards compatible

At Stainless we design SDKs that gracefully evolve with your API. In particular, we pay close attention to forwards compatibility in our SDKs. Non-breaking changes to your API shouldn’t become breaking changes when it's time to update your SDKs!

Achieving this can be surprisingly difficult and problems arise in unexpected places. Let’s take a look at a recent seemingly small issue that came up when designing our Java SDK.

An example API

Suppose we’re designing a Java SDK for a Pet Store API. We’d probably start with a class and builder for each API object. One class might look like so:

public final class PetParam {
    private final String name;
    private final long tagId;

    private PetParam(String name, long tagId) {
        this.name = name;
        this.tagId = tagId;
    }

    public static Builder builder() {
        return new Builder();
    }

    public static final class Builder {

        private Builder() {}

        private String name;
        private Long tagId;

        public Builder name(String name) {
            this.name = checkNotNull(name);
            return this;
        }

        public Builder tagId(long tagId) {
            this.tagId = tagId;
            return this;
        }

        public PetParam build() {
            return new PetParam(checkNotNull(name), checkNotNull(tagId));
        }
    }
}

Suppose we ship the SDK and users are happily using it. Then one day we decide to make tagId optional.

“No problem,” we think. Let’s just make the tagId parameter boxed so that it can accept null:

 public final class PetParam {

     private final String name;
-    private final long tagId;
+    private final Long tagId;

-    private PetParam(String name, long tagId) {
+    private PetParam(String name, Long tagId) {
         this.name = name;
         this.tagId = tagId;
     }

     // ...

     public static final class Builder {

         // ...

-        public Builder tagId(long tagId) {
+        public Builder tagId(Long tagId) {
             this.tagId = tagId;
             return this;
         }

         public PetParam build() {
-            return new PetParam(checkNotNull(name), checkNotNull(tagId));
+            return new PetParam(checkNotNull(name), tagId);
         }
     }
 }

Seems reasonable right? There are actually a couple of problems.

Problem 1: Java primitive conversions

Imagine a user wrote the following code before our change:

PetParam pet = PetParam.builder()
    .name("Nickel")
    .tagId(42)
    .build();

We might expect the user’s code to keep working. We just widened the type we accept, right? Wrong.

PetParam pet = PetParam.builder()
    .name("Nickel")
    .tagId(42)
        // ^ error: incompatible types: int cannot be converted to Long
    .build();

What happened here? It turns out that Java performs automatic conversion between:

Primitive types (e.g. int to long)
Unboxed and boxed (e.g. long to Long)

But it cannot do both at the same time! Some more examples:

void unboxed(long value) {}
void boxed(Long value) {}

// ...

// Success: `int` converted to `long` automatically
unboxed(42);

// Success: `long` converted to `Long` automatically
boxed(42L);

// Failed: cannot automatically box AND convert primitive type
boxed(42);
   // ^ error: incompatible types: int cannot be converted to Long

Problem 2: Binary compatibility

Imagine a user wrote the following code before our change:

PetParam pet = PetParam.builder()
    .name("Nickel")
    .tagId(42L)
    .build();

Clearly it doesn’t run into Problem 1, so are we in the clear? Nope.

Suppose this code was released as a library called nickel-lib prior to the new SDK version. If another user:

Depends on both the SDK (directly) and nickel-lib
Upgrades the direct SDK dependency version without also upgrading nickel-lib
Dependency resolution choose the newer version

Then nickel-lib would encounter a NoSuchMethodError the next time the user runs their code. Why?

Although tagId(42L) is source-compatible with both tagId(long) and tagId(Long), tagId(long) and tagId(Long) have different JVM type signatures, and just one of those is chosen when compiling tagId(42L).

The result is that nickel-lib's *.class file, compiled before our change, is still trying to look up tagId(long) after our change, but it no longer exists. Our change is not binary compatible.

Solution

So what’s an SDK engineer to do?

When making our change, we should have preserved the original signature like so:

 public static final class Builder {

     // ...

     public Builder tagId(long tagId) {
-        this.tagId = tagId;
-        return this;
+        // Box the `tagId` for the user.
+        return tagId((Long) tagId);
     }

+    public Builder tagId(Long tagId) {
+        this.tagId = tagId;
+        return this;
+    }

     // ...
 }

This fixes both problems:

PetParam pet = PetParam.builder()
    .name("Nickel")
    // Success:
    // 1. `int` converted to `long` automatically
    // 2. Then boxing happens internally
    // Plus, `tagId(long)` still exists so we preserve binary compatibility
    .tagId(42)
    .build();

And there’s an additional benefit even when a parameter is nullable from the beginning. Users can pass literal int for nullable Long fields without trouble.

‍

Get updates from Stainless

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form.

Did this happen at Stainless?

We actually caught this problem during design review, so the issue never made its way to customers!

At Stainless, we obsess over these types of details like no one else—so you can ship beautiful, reliable code that your users love. Start shipping SDKs this afternoon with the Stainless Studio.