unit of intensive care (2020.12 mixtape)

• Spotify: https://open.spotify.com/playlist/1mQ6JuDwWlYvuqqji53E4N?si=AopTuNoTTyCVBcOraXiZ-w
• YouTube: https://www.youtube.com/playlist?list=PLSUh6oJ5ZotW71OG0WR0XTJl3IlcJGNIZ

4h

auto publish sbt plugin from GitHub Actions

This is a GitHub Actions version of auto publish sbt plugin from Travis CI.

In this post, we’ll try to automate the release of an sbt plugin using Ólaf’s olafurpg/sbt-ci-release. The README of sbt-ci-release covers the use case for a library published to Sonatype OSS. Read it thoroughly since this post will skip over the details that do not change for publishing sbt plugins.

Automated release in general is a best practice, but there’s one benefit specifically for sbt plugin releases. Using this setup allows multiple people to share the authorization to release an sbt plugin without adding them to Bintray sbt organization. This is useful for plugins maintained at work.

scopt 4

This post was first published in December 2018 together with 4.0.0-RC2. It’s updated to reflect the changes made in November 2020 for 4.0.0.

You can skip to the readme, if you’re in a hurry.

To try new scopt 4.0.0:

libraryDependencies += "com.github.scopt" %% "scopt" % "4.0.0"

scopt 4.0.0 is cross published for the following build matrix:

scopt is a little command line options parsing library. scopt started its life in 2008 as aaronharnly/scala-options based loosely on Ruby’s OptionParser. scopt 2 added immutable parsing, and scopt 3 cleaned up the number of methods by introducing Read typeclass.

Weehawken-Lang1

about Weehawken-Lang

It’s a strange time we live in. We can’t just meet up and catch up and talk about coding. This also opens an opportunity to think more virtually about the idea of meetups.

I want to start Weehawken-Lang, a virtual meetup group about programming languages and tooling design (compilers, interpreters, build tools etc). It aims to be a casual place where people with different language backgrounds can exchange ideas about programming languages.

sbt 1.4.4

I’m happy to announce sbt 1.4.4 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.4

How to upgrade

Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.

In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:

sbt.version=1.4.4

This mechanism allows that sbt 1.4.4 is used only for the builds that you want.

Highlights

• Updates SemanticDB to 4.4.0 to support Scala 2.13.4 #6148 by @adpi2
• Fixes sbt plugin cross building fix #6091/#6151 by @xuwei-k and @eatkins
• Fixes scala-compiler not included into the metabuild classpath #6146 by @eatkins
• Fixes UTF-8 handling in shell and console #6106 by @eatkins
• Fixes macro occasionally dropping expressions trying to work around Scala compiler displaying “a pure expression does nothing” when sbt is really doing something #6158 by @eed3si9n
• Global / localCacheDirectory for remote caching #6155 by @eed3si9n
• Adds system property sbt.build.onchange for onChangedBuildSource #6099 by @xirc

For more details please see https://github.com/sbt/sbt/releases/tag/v1.4.4

sbt 1.4.3

I’m happy to announce sbt 1.4.3 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.3

How to upgrade

Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/. This installer includes the sbtn binary.

In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:

sbt.version=1.4.3

This mechanism allows that sbt 1.4.3 is used only for the builds that you want.

Highlights

• Updates to Coursier 2.0.6 #6036 by @jtjeferreira
• Fixes IntelliJ import on Windows #6051 by @eatkins
• Fixes the dependency resolution in metabuild #6085 by @eed3si9n
• Removes “duplicate compilations” assertion to work around Play issue zinc#940 by @johnduffell
• Fixes GC monitor warnings #6082 by @nafg

For more details please see https://github.com/sbt/sbt/releases/tag/v1.4.3

sbt 1.4.2

I’m happy to announce sbt 1.4.2 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.2

How to upgrade

Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/. This installer includes the sbtn binary.

In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:

sbt.version=1.4.2

This mechanism allows that sbt 1.4.2 is used only for the builds that you want.

Highlights

• Remote caching is now content-based #6026 by @eed3si9n
• installSbtn wizard for installing sbtn and completions #6023 by @eatkins
• Fixes memory leak during task evaluation #6001 by @eatkins
• Various read line and character handling fixes by @eatkins
• Various BSP fixes by @adpi2

For more details please see https://github.com/sbt/sbt/releases/tag/v1.4.2

remote caching sbt builds with Bintray

The feature in sbt and Zinc 1.4.x that I spent most amount of time and energy probably is the virtualization of file, and lifting out timestamps. Combined together, we can liberate the Zinc state from machine-specificity and time, and become the foundation we lay towards building incremental remote caching for Scala. I blogged about this in cached compilation for sbt. This is part 2.

Now that sbt 1.4.x is out, there is a growing interest in this feature among people who want to try this out.

sbt 1.4.1

I’m happy to announce sbt 1.4.1 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.1

How to upgrade

Download the official sbt launcher from SDKMAN or download from https://www.scala-sbt.org/download.html. This installer includes the sbtn binary.

In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:

sbt.version=1.4.1

This mechanism allows that sbt 1.4.1 is used only for the builds that you want.

Highlights

• Various read line and character handling fixes by @eatkins, including sbt new not echoing back the characters
• Fixes Scala 2.13-3.0 sandwich support for Scala.JS #5984 by @xuwei-k
• Fixes shellPrompt and release* keys warning on build linting #5983/#5991 by @xirc and @eed3si9n
• Improves plugin command output by grouping by subproject #5932 by @aaabramov

For more details please see https://github.com/sbt/sbt/releases/tag/v1.4.1

virtualizing a hackathon at ScalaMatsuri 2020

Here’s a report of running a virtual hackathon at ScalaMatsuri Day 2 Unconference. Someone proposed it for the Unconference, and I volunteered to be a facilitator on the day, so I went in without preparation. I booked the time originally for 4h (noon - 4pm JST, 11pm - 3am EDT) but it was successful so it got extended after some coffee break.

One thing I emphasize is The Law of Two Feet:

Equality in Scala

I gave a talk at ScalaMatsuri on ‘Equality in Scala’

• slides: https://eed3si9n.com/equalty
• pdf

sbt 1.4.0

Hi everyone. On behalf of the sbt project, I am happy to announce sbt 1.4.0. This is the fourth feature release of sbt 1.x, a binary compatible release focusing on new features. sbt 1.x is released under Semantic Versioning, and the plugins are expected to work throughout the 1.x series.

The headline features of sbt 1.4.0 are:

• build server protocol (BSP) support
• sbtn: a native thin client for sbt
• build caching
• ThisBuild / versionScheme to take the guessing out of eviction warning

sbt 1.4.0-RC2

Hi everyone. On behalf of the sbt project, I am happy to announce sbt 1.4.0-RC2. This is the fourth feature release of sbt 1.x, a binary compatible release focusing on new features. sbt 1.x is released under Semantic Versioning, and the plugins are expected to work throughout the 1.x series.

• If no serious issues are found by Saturday, October 3rd 2020, 1.4.0-RC2 will become 1.4.0 final.
• If no serious issues are found by Saturday, September 19th 2020, 1.4.0-RC1 will become 1.4.0 final.

The headline features of sbt 1.4.0 are:

• build server protocol (BSP) support
• sbtn: a native thin client for sbt
• build caching
• ThisBuild / versionScheme to take the guessing out of eviction warning

metropolitan house supply (2020.09 mixtape)

• Spotify: https://open.spotify.com/playlist/62RqkwMDQ6k8o88fjE7Oc2?si=8tT0-NnoRUK3E0-Kug1akg
• YouTube: https://www.youtube.com/playlist?list=PLSUh6oJ5ZotVrO_XPe6q_yrfcpYFegLwG

4h 29m

parallel cross building sandwich

This is part 4 of the post about sbt-projectmatrix, an experimental plugin that I’ve been working to improve the cross building in sbt. Here’s part 1, part 2, and part 3. I’ve just released 0.6.0.

recap: building against multiple Scala versions

After adding sbt-projectmatrix to your build, here’s how you can set up a matrix with two Scala versions.

ThisBuild / organization := "com.example"
ThisBuild / scalaVersion := "2.12.12"
ThisBuild / version      := "0.1.0-SNAPSHOT"

lazy val core = (projectMatrix in file("core"))
  .settings(
    name := "core"
  )
  .jvmPlatform(scalaVersions = Seq("2.12.12", "2.13.3"))

This will create subprojects for each scalaVersion. Unlike ++ style stateful cross building, these will build in parallel. This part has not changed.

joining Twitter

I’m excited to announce that I’m joining Twitter’s Build Team to work on the next generation of efficient build systems supporting thousands of Twitter developers worldwide. Today’s my first day.

This is the team that developed monorepo build tool Pants, and is transitioning to migrate the flock to Bazel. This presented a unique opportunity for me to work with a team of people passionate about developer experience and productivity, and I’m looking forward to getting to know the team, and learning the new challenges.

sbt-buildinfo 0.10.0

I’m happy to announce sbt-buildinfo 0.10.0. sbt-buildinfo is a small sbt plugin to generate BuildInfo object from your build definitions.

Since the last feature release was in 2018, there have been some pending contributions. I think the important thing is that it compiles with -Xlint and -Xfatal-warnings on both Scala 2.13.3 and 2.12.12.

auto publish sbt plugin from Travis CI

In this post, we’ll try to automate the release of an sbt plugin using Ólafur’s olafurpg/sbt-ci-release. The README of sbt-ci-release covers the use case for a library published to Sonatype OSS. Read it thoroughly since this post will skip over the details that do not change for publishing sbt plugins.

Automated release in general is a best practice, but there’s one benefit specifically for sbt plugin releases. Using this setup allows multiple people to share the authorization to release an sbt plugin without adding them to Bintray sbt organization. This is useful for plugins maintained at work.

pb by j

Spotify: https://open.spotify.com/playlist/5BkvksBTbrngqXxhF6wUAc?si=Kj8TrnhRQICDTgadvYYt2w YouTube: https://www.youtube.com/playlist?list=PLSUh6oJ5ZotXfMWZPzm6K_L0CRPgFhxGh 2h 4m

Defx: file explorer plugin for Neovim

sbt 1.3.13

I’m happy to announce sbt 1.3.13 patch release. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.3.13.

Special thanks to Scala Center. It takes time to review bug reports, pull requests, make sure contributions land to the right places, and Scala Center sponsored me to do maintainer tasks for sbt during June.

reentrance protocol (2020.06 mixtape)

• Spotify: https://open.spotify.com/playlist/4fAHIQhqD4GdaqoCZVrULG?si=crwaw9y-Q-yfqwErXiTqJg
• YouTube: https://www.youtube.com/playlist?list=PLSUh6oJ5ZotXZXAHittTWaNgO3uGdvls7

3h 59m

sbt 1.3.12

I’m happy to announce sbt 1.3.12 patch release. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.3.12.

Special thanks to Scala Center. It takes time to review bug reports, pull requests, make sure contributions land to the right places, and Scala Center sponsored me to do maintainer tasks for sbt during May. Darja + whole Scala Center crew have been chill to work with.

Change since sbt 1.3.11

There was a regression in sbt 1.3.11 around the launcher integration, which showed up as repositories file getting ignored. sbt 1.3.12 fixes it. #5583

sbt 1.3.11

I’m happy to announce sbt 1.3.11 patch release. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.3.11.

Special thanks to Scala Center. It takes time to review bug reports, pull requests, make sure contributions land to the right places, and Scala Center sponsored me to do maintainer tasks for sbt during May. Darja + whole Scala Center crew have been chill to work with.

How to upgrade

Normally changing the project/build.properties to

sbt.version=1.3.11

would be ok. However, given that the release may contain fixes to scripts and also because your initial resolution would be faster with *.(zip|tgz|msi) that contains all the JAR files, we recommend you use the installer distribution. They will be available from SDKMAN etc:

Jar Jar Abrams

Jar Jar Abrams is an experimental Scala extension of Jar Jar Links, a utility to shade Java libraries.

For library authors, the idea of other library is a double-edged sword. On one hand, using other libraries avoids unnecessary duplication of work, not using other libraries is almost hypocritical. On the other hand, each library you add would add a transitive dependency to your users, increasing the possibility of conflict. This is partly due to the fact that within a single running program you can one have one version of a library.

Selective functor in sbt

In sbt core concepts talks I’ve been calling sbt a casually functional build tool. Two hallmarks of functional programming are that it uses immutable data structure instead of mutation, and that it gives attention to when and how effects are handled.

settings and tasks

From this perspective, we can think of setting expressions and tasks to be those two things:

• Settings form an immutable graph in a build.
• Tasks represent effects.

Anonymous settings are represented using Initialize[A], which looks like this:

keeping promise in sbt

build.sbt is a DSL for defining a task graph to be used for automatic parallel processing. The message passing among the tasks are expressed using something.value macro, which encodes Applicative composition (task1, task2) mapN { case (t1, t2) => .... }.

One mechanism I’ve been thinking about is allowing some long-running task1 to communicate with task2 midway.

Normally, we would break down task1 into two subtasks. But it might not be as straight-forward to implement such thing. For example, how would be tell Zinc to compile something halfway, and resume later? Or tell Coursier to resolve, but fetch later?

cached compilation for sbt

sbt 1.3.10

I’m happy to announce sbt 1.3.10 patch release. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.3.10

How to upgrade

Normally changing the project/build.properties to

sbt.version=1.3.10

would be ok. However, given that the release may contain fixes to scripts and also because your initial resolution would be faster with *.(zip|tgz|msi) that contains all the JAR files, we recommend you use the installer distribution. They will be available from SDKMAN etc.

Notes about Homebrew

Homebrew maintainers have added a dependency to JDK 13 because they want to use more brew dependencies brew#50649. This causes sbt to use JDK 13 even when java available on PATH is JDK 8 or 11.

Zinc 1.4.0-M1

I’ve just released Zinc 1.4.0-M1. Note this is a beta release and it won’t be compatible with future 1.4.x, but I chose a commit fairly close to 1.3.x so it should be usable.

• Cross builds Zinc to Scala 2.12 and 2.13 zinc#754 by @eed3si9n
• Upgrades ScalaPB to 0.9.3 zinc#713 by @slandelle
• Replaces ZipUtils usage of deprecated java.util.Date with java.time zinc#714 by @slandelle

Zinc is an incremental compiler for Scala. Though Zinc is capable of compiling Scala 2.10 ~ 2.13 and Dotty, thus far Zinc itself has been implemented using Scala 2.12. This is fine for sbt 1.x, which is also implemented in Scala 2.12, but there’s been requests to cross build Zinc for 2.13.