search term:

sbt 1.3.12

I’m happy to announce sbt 1.3.12 patch release. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.3.12.

Special thanks to Scala Center. It takes time to review bug reports, pull requests, make sure contributions land to the right places, and Scala Center sponsored me to do maintainer tasks for sbt during May. Darja + whole Scala Center crew have been chill to work with.

Change since sbt 1.3.11

There was a regression in sbt 1.3.11 around the launcher integration, which showed up as repositories file getting ignored. sbt 1.3.12 fixes it. #5583

Read More…

sbt 1.3.11

I’m happy to announce sbt 1.3.11 patch release. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.3.11.

Special thanks to Scala Center. It takes time to review bug reports, pull requests, make sure contributions land to the right places, and Scala Center sponsored me to do maintainer tasks for sbt during May. Darja + whole Scala Center crew have been chill to work with.

How to upgrade

Normally changing the project/build.properties to

Read More…

Jar Jar Abrams

Jar Jar Abrams is an experimental Scala extension of Jar Jar Links, a utility to shade Java libraries.

For library authors, the idea of other library is a double-edged sword. On one hand, using other libraries avoids unnecessary duplication of work, not using other libraries is almost hypocritical. On the other hand, each library you add would add a transitive dependency to your users, increasing the possibility of conflict. This is partly due to the fact that within a single running program you can one have one version of a library.

Read More…

Selective functor in sbt

In sbt core concepts talks I’ve been calling sbt a casually functional build tool. Two hallmarks of functional programming are that it uses immutable data structure instead of mutation, and that it gives attention to when and how effects are handled.

settings and tasks

From this perspective, we can think of setting expressions and tasks to be those two things:

  • Settings form an immutable graph in a build.
  • Tasks represent effects.

Anonymous settings are represented using Initialize[A], which looks like this:

Read More…

keeping promise in sbt

build.sbt is a DSL for defining a task graph to be used for automatic parallel processing. The message passing among the tasks are expressed using something.value macro, which encodes Applicative composition (task1, task2) mapN { case (t1, t2) => .... }.

One mechanism I’ve been thinking about is allowing some long-running task1 to communicate with task2 midway.

promise

Normally, we would break down task1 into two subtasks. But it might not be as straight-forward to implement such thing. For example, how would be tell Zinc to compile something halfway, and resume later? Or tell Coursier to resolve, but fetch later?

Read More…

cached compilation for sbt

The notion of cached compilation or remote cache has been around for a while now, but often it required the setup has not been easy. If we can bake build-as-function feature into basic tool chain such as Zinc and sbt, Scala community in general can benefit from it to speed up the build. Even for open source projects, if Travis CI publishes into Bintray or something, the contributors might be able to resume compilation off of the last build. The PR for sbt change is sbt/sbt#5534, and the virtualization change in Zinc is sbt/zinc#712.

sbt 1.3.10

I’m happy to announce sbt 1.3.10 patch release. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.3.10

How to upgrade

Normally changing the project/build.properties to

sbt.version=1.3.10

would be ok. However, given that the release may contain fixes to scripts and also because your initial resolution would be faster with *.(zip|tgz|msi) that contains all the JAR files, we recommend you use the installer distribution. They will be available from SDKMAN etc.

Read More…

Zinc 1.4.0-M1

I’ve just released Zinc 1.4.0-M1. Note this is a beta release and it won’t be compatible with future 1.4.x, but I chose a commit fairly close to 1.3.x so it should be usable.

Zinc is an incremental compiler for Scala. Though Zinc is capable of compiling Scala 2.10 ~ 2.13 and Dotty, thus far Zinc itself has been implemented using Scala 2.12. This is fine for sbt 1.x, which is also implemented in Scala 2.12, but there’s been requests to cross build Zinc for 2.13.

Read More…

parallel cross building, part 3

This is part 3 of the post about sbt-projectmatrix, an experimental plugin that I’ve been working to improve the cross building in sbt. Here’s part 1 and part 2. I’ve just released 0.5.0.

recap: building against multiple Scala versions

After adding sbt-projectmatrix to your build, here’s how you can set up a matrix with two Scala versions.

ThisBuild / organization := "com.example"
ThisBuild / scalaVersion := "2.12.10"
ThisBuild / version      := "0.1.0-SNAPSHOT"

lazy val core = (projectMatrix in file("core"))
  .settings(
    name := "core"
  )
  .jvmPlatform(scalaVersions = Seq("2.12.10", "2.11.12"))

    Read More…

6 years at Lightbend

I joined Lightbend (then Typesafe) in March, 2014. After six incredible years April 7, 2020 was my last day. I am grateful that I got the opportunity to work with an amazing group of people at Lightbend, partners, customers, and various people I got to meet at conferences. Looking back, before COVID-19 times, it’s almost surreal that I was flying to Europe, Asia, and North America every few months to attend conferences and tech meetings.

Read More…

user-land compiler warnings in Scala, part 2

Last week I wrote about #8820, my proposal to add user-land compiler warnings in Scala. The example I had was implementing ApiMayChange annotation.

package foo

import scala.annotation.apiStatus, apiStatus._

@apiStatus(
  "should DSL is incubating, and future compatibility is not guaranteed",
  category = Category.ApiMayChange,
  since = "foo-lib 1.0",
  defaultAction = Action.Warning,
)
implicit class ShouldDSL(s: String) {
  def should(o: String): Unit = ()
}

    Read More…

bianlu plant-based hotpot

serves 4?

ingredients

  • 1 head of napa cabbage
  • 100 ml of semame oil
  • 5 dried shiitake
  • some dried kombu kelp
  • 160g vermicelli bean thread
  • frozen meatless meatballs
  • kosher salt
  • chili oil, chili oil with fried garlic, or chili powder
  • (optional) cooked white rice

steps

  1. put cold water in a large dutch oven with 5 dried shiitake and similar amount of dried kombu kelp. let it sit for a few hours covered.

    Read More…

user-land compiler warnings in Scala

As a library author, I’ve been wanting to tag methods in Scala that can trigger custom warnings or compiler errors. Why would I want to intentionally cause a compiler error? One potential use case is displaying a migration message for a removed API.

For example, if you try to use <<= in sbt 1.3.8 you’d get the following error on load:

Read More…

equal protection under Eq law

The relationship given to Int and Long should be exactly the same as the relationship third-party library like Spire can write UInt or Rational with the first-class numeric types.

  • We should make 1 == 1L an error under strictEquality
  • We should allow custom types to participate in constant expression conversion using FromDigits

liberty, equality, and boxed primitive types

I want to understand how equality works in Scala. It’s a complicated topic that’s been going on for ten years.

Major concerns are:

  • null
  • Unboxed number types
  • Boxed number types
  • Reference types
  • Collections (especially of F[+A])

Understanding equality means knowing how these combinations are compared.

Scala Language Specification

The language spec provides some hints, although it does not have the full information. Chapter 12 contains the definition of Any as follows:

Read More…

Giter8 0.12.0

giter8.version

I added a small app called giter8-launcher for Giter8 0.12.0. The purpose of the app is to make the behavior of the Giter8 template more predictable. Today, template authors may create a template for some version of Giter8 X, but the users might use some other version of Giter8 Y that ships with “sbt new.”

One of the neat ideas about sbt is that no matter what version of sbt script users might have it installed, the core sbt version is specified by the build author using project/build.properties file. This significantly reduces the it-only-works-on-my-machine problem. giter8-launcher is analogous to sbt’s sbt-launcher. giter8-launcher clones the template and reads project/build.properties file to determine the Giter8 version to render the template.

Template authors can now specify the Giter8 version in project/build.properties file as:

giter8.version=0.12.0
Read More…

parallel cross building with VirtualAxis

This is part 2 of the post about sbt-projectmatrix, an experimental plugin that I’ve been working to improve the cross building in sbt. Here’s part 1. I’ve just released 0.4.0.

recap: building against multiple Scala versions

After adding sbt-projectmatrix to your build, here’s how you can set up a matrix with two Scala versions.

ThisBuild / organization := "com.example"
ThisBuild / scalaVersion := "2.12.10"
ThisBuild / version      := "0.1.0-SNAPSHOT"

lazy val core = (projectMatrix in file("core"))
  .settings(
    name := "core"
  )
  .jvmPlatform(scalaVersions = Seq("2.12.10", "2.11.12"))

    Read More…

Pamflet 0.8.2

Pamflet is a publishing application for short texts, particularly user documentation of open-source software.

Pamflet 0.8.2 updates its monospace typeface to SFMono, and undoes the incidental pink color that got introduced when I migrated from Blueprint to Bootstrap.

Read More…

semantics of dependency resolvers

The semantics of a dependency resolver determine the concrete classpath based on the user-specified dependency constraints. Typically the differences in the details manifest as different way the version conflicts are resolved.

  • Maven uses nearest-wins strategy, which could downgrade transitive dependencies
  • Ivy uses latest-wins strategy
  • Coursier generally uses latest-wins strategy, but it’s tries to enforce version range strictly
  • Ivy’s version range handling goes to the Internet, which makes the build non-repeatable
  • Coursier orders version string completely differently from Ivy

parallel cross building using sbt-projectmatrix

Last year I wrote an experimental sbt plugin called sbt-projectmatrix to improve the cross building in sbt. I’ve just released 0.2.0.

building against multiple Scala versions

After adding sbt-projectmatrix to your build, here’s how you can set up a matrix with two Scala versions.

ThisBuild / organization := "com.example"
ThisBuild / scalaVersion := "2.12.8"
ThisBuild / version      := "0.1.0-SNAPSHOT"

lazy val core = (projectMatrix in file("core"))
  .settings(
    name := "core"
  )
  .jvmPlatform(scalaVersions = Seq("2.12.8", "2.11.12"))

    Read More…

splitting git repo

split a subdirectory into a new repo (simple case)

git clone --no-hardlinks --branch master originalRepoURL childRepo
cd childRepo
git filter-branch --prune-empty --subdirectory-filter path/to/keep master
git remote remove origin
git prune
git gc --aggressive

Change originalRepoURL, master, and path/to/keep to appropriate values. Use -- --all to handle all branches.

split a subdirectory into a new repo (complex case)

In case you have multiple paths you want to filter, you need to use --index-filter together with GNU xargs and GNU sed available via brew install gnu-sed findutils.

Read More…

all your JDKs on Travis CI using SDKMAN!

This is a second post on installing your own JDKs on Travis CI. Previously I’ve written about jabba.

Today, let’s look at SDKMAN!, an environment manager written by Marco Vermeulen (@marc0der) for JDKs and various tools on JVM, including Groovy, Spark, sbt, etc.

AdoptOpenJDK 11 and 8

  • Update 2020-09-23: Updated the regex of version number.
  • Update 2019-11-06: Added sdkman_auto_selfupdate to workaround the update prompt blocking the CI. Also it adds || true on the sdk install line.
  • Update 2019-07-08: Updated the script to detect patch version. See GitHub for the older version.

Here’s how we can use SDKMAN! on Travis CI to cross build using AdoptOpenJDK 11 and 8:

Read More…

'Ancillary Justice' by Ann Leckie

Ancillary Justice (Imperial Radch, #1)Ancillary Justice by Ann Leckie
My rating: 4 of 5 stars

‘Ancillary Justice’ is written in a first-person narrative as the AI of thousands of years old starship Justice of Toren of The Radchaai Empire. Another interesting part is since the Radchaai language is genderless, the narrator constantly describes everyone as “she,” but later you might discover that some character might be he. There are other small details here and there that narrator might say, but you start to question it as the story develops. In other words, Ann Leckie has done a stylish job of show-and-not-tell about the world that she’s built.

Read More…

Pamflet 0.8.0

Over the holiday break I’ve implemented left TOC for Pamflet, and released it as Pamflet 0.8.0.

Pamflet is a publishing application for short texts, particularly user documentation of open-source software.

Read More…

masking scala.Seq

As of Scala 2.13.0-M5, it’s planned that scala.Seq will change from scala.collection.Seq to scala.collection.immutable.Seq. Scala 2.13 collections rework explains a bit about why it’s been non-immutable historically. Between the lines, I think it’s saying that we should celebrate that scala.Seq will now be immutable out of the box.

Defaulting to immutable sequence would be good for apps and fresh code. The situation is a bit more complicated for library authors.

Read More…

making conference a safer space for women

We need to change the culture around tech conferences to improve the inclusion of women (and people from other backgrounds too!). For that, there needs to be clear signaling and communication about two basic issues

  1. No, it’s not ok to hit on women at a conference.
  2. Assume technical competence, and treat women as professional peers. These points should be communicated over and over at each conference before the keynote takes place, and before socializing hours.

'Red Mars' by Kim Stanley Robinson

Red Mars (Mars Trilogy, #1)Red Mars by Kim Stanley Robinson
My rating: 4 of 5 stars

Red Mars is a science fiction classic written in 1992. This book is everything I wanted and more, starting with first hundred astronauts and cosmonauts migrating to Mars to build the infrastructure such that more people can migrate. By the middle of the book, thousands of people migrate to Mars. The book explores various dimensions the epic project, not just technological challenges, but psychological effects, personality differences, and political interests by superpowers. The geography of Mars is written in vivid, majestic details.

Read More…

super shell for sbt

I’ve implemented “super shell” feature for sbt over the weekend. The idea is to take over the bottom n lines of the termnial, and display the current tasks in progress.

the limitation of using log as status report

Logs are useful in many situations, and sometimes it’s the only viable tool to find out what’s going on. But on a console app like sbt, using logs to tell the build user what’s going on doesn’t always work.

Read More…

stricter Scala with -Xlint, -Xfatal-warnings, and Scalafix

Compile, or compile not. There’s no warning. Two of my favorite Scala compiler flags lately are "-Xlint" and "-Xfatal-warnings". Here is an example setting that can be used with subprojects:

ThisBuild / organization := "com.example"
ThisBuild / version      := "0.1.0-SNAPSHOT"
ThisBuild / scalaVersion := "2.12.6"

lazy val commonSettings = List(
  scalacOptions ++= Seq(
    "-encoding", "utf8",
    "-deprecation",
    "-unchecked",
    "-Xlint",
    "-feature",
    "-language:existentials",
    "-language:experimental.macros",
    "-language:higherKinds",
    "-language:implicitConversions",
    "-Ypartial-unification",
    "-Yrangepos",
  ),
  scalacOptions ++= (scalaVersion.value match {
    case VersionNumber(Seq(2, 12, _*), _, _) =>
      List("-Xfatal-warnings")
    case _ => Nil
  }),
  Compile / console / scalacOptions --= Seq("-deprecation", "-Xfatal-warnings", "-Xlint")
)

lazy val foo = (project in file("foo"))
  .settings(
    commonSettings,
    name := "foo",  
  )

    Read More…

git gone: cleaning stale local branches

Working with GitHub and pull requests a lot, I end up accumulating stale branches that are no longer needed. In this post, we will look at how to clean the stale local branches.

There are mainly two strategies:

  • Pick a “master” branch, and delete what’s merged to it
  • Assuming branches are deleted first on GitHub, delete local branches that no longer exists on remote “origin”

Erik Aybar’s Git Tip: Deleting Old Local Branches takes the second approach.

Read More…

console games in Scala

I’ve been thinking about rich console applications, the kind of apps that can display things graphically, not just appending lines at the end. Here are some info, enough parts to be able to write Tetris.

ANSI X3.64 control sequences

To display some text at an arbitrary location on a termial screen, we first need to understand what a terminal actually is. In the middle of 1960s, companies started selling minicomputers such as PDP-8, and later PDP-11 and VAX-11. These were of a size of a refrigerator, purchased by “computer labs”, and ran operating systems like RT-11 and the original UNIX system that supported up many simultaneous users (12 ~ hundreds?). The users connected to a minicomputer using a physical terminal that looks like a monochrome screen and a keyboard. The classic terminal is VT100 that was introduced in 1978 by DEC.

Read More…

all your JDKs on Travis CI using jabba

Whether you want to try using OpenJDK 11-ea, GraalVM, Eclipse OpenJ9, or you are stuck needing to build using OpenJDK 6, jabba has got it all. jabba is a cross-platform Java version manager written by Stanley Shyiko (@shyiko).

AdoptOpenJDK 8 and 11

Here’s how we can use jabba on Travis CI to cross build using AdoptOpenJDK 8 and 11:

sudo: false
dist: trusty
group: stable

language: scala

scala:
  - 2.12.7

env:
  global:
    - JABBA_HOME=/home/travis/.jabba

matrix:
  include:
  - env:
      - TRAVIS_JDK=adopt@1.8.192-12
  - env:
      - TRAVIS_JDK=adopt@1.11.0-1

before_install:
  - curl -sL https://raw.githubusercontent.com/shyiko/jabba/0.11.0/install.sh | bash && . ~/.jabba/jabba.sh

install:
  - $JABBA_HOME/bin/jabba install $TRAVIS_JDK
  - unset _JAVA_OPTIONS
  - export JAVA_HOME="$JABBA_HOME/jdk/$TRAVIS_JDK" && export PATH="$JAVA_HOME/bin:$PATH" && java -Xmx32m -version

script: sbt -Dfile.encoding=UTF8 -J-XX:ReservedCodeCacheSize=256M ++$TRAVIS_SCALA_VERSION! test

before_cache:
  - find $HOME/.ivy2 -name "ivydata-*.properties" -delete
  - find $HOME/.sbt  -name "*.lock"               -delete

cache:
  directories:
    - $HOME/.ivy2/cache
    - $HOME/.sbt/boot
    - $HOME/.jabba/jdk

    Read More…

bringing back power assert with Expecty

Last week I wrote about using source dependencies with sbt-sriracha for testing purpose. This week we’ll look into using Expecty to do power assert.

Power assert (or power assertion) is a variant of assert(...) function that that prints out detailed error message automatically. It was originally implemented by Peter Niederwieser (@pniederw) for Spock, and in 2009 it was merged into Groovy 1.7. Power assert has spread to Ruby, JavaScript, Rust, etc.

Read More…

hot source dependencies using sbt-sriracha

Source dependencies is one of features that existed in sbt since ever, but hasn’t been documented well.

immutable source dependency

Here’s how to declare source dependency to the latest commit for scopt commandline option parsing library.

lazy val scoptJVMRef = ProjectRef(uri("git://github.com/scopt/scopt.git#c744bc48393e21092795059aa925fe50729fe62b"), "scoptJVM")

ThisBuild / organization := "com.example"
ThisBuild / scalaVersion := "2.12.2"

lazy val root = (project in file("."))
  .dependsOn(scoptJVMRef)
  .settings(
    name := "Hello world"
  )

    Read More…

detecting Java version from Bash

Yesterday I wrote about cross JVM testing using Travis CI.

testing Scala apps on macOS using Travis CI

Here’s how we can test Scala apps on macOS using Travis CI. This is adapted from Lars and Muuki’s method: Testing Scala programs with Travis CI on OS X

dist: trusty

language: scala

matrix:
  include:
    ## build using JDK 8, test using JDK 8
    - script:
        - sbt universal:packageBin
        - cd citest && ./test.sh
      jdk: oraclejdk8

    ## build using JDK 8, test using JDK 8, on macOS
    - script:
        - sbt universal:packageBin
        - cd citest && ./test.sh
      ## https://github.com/travis-ci/travis-ci/issues/2316
      language: java
      os: osx
      osx_image: xcode9.2

    ## build using JDK 8, test using JDK 9
    - script:
        - sbt universal:packageBin
        - jdk_switcher use oraclejdk9
        - cd citest && ./test.sh
      jdk: oraclejdk8

    ## build using JDK 8, test using JDK 10
    - script:
        - sbt universal:packageBin
        - citest/install-jdk10.sh
        - cd citest && ./test.sh
      jdk: oraclejdk8

scala:
  - 2.10.7

before_install:
  # https://github.com/travis-ci/travis-ci/issues/8408
  - unset _JAVA_OPTIONS
  - if [[ "$TRAVIS_OS_NAME" = "osx" ]]; then
      brew update;
      brew install sbt;
    fi

cache:
  directories:
    - $HOME/.ivy2/cache
    - $HOME/.sbt/boot

before_cache:
  - find $HOME/.ivy2 -name "ivydata-*.properties" -delete
  - find $HOME/.sbt  -name "*.lock"               -delete

    Read More…

cross JVM testing using Travis CI

Oracle is moving to ship non-LTS JDK every 6 months, and LTS JDK every 3 years. Also it’s converging to OpenJDK. In this scheme, JDK 9 will be EOL in March 2018; JDK 10 will come out in March 2018, and EOL in September 2018; and LTS JDK 11 that replaces JDK 8 in September 2018 will stay with us until 2021.

As we will see quick succession of JDKs in the upcoming months, here’s a how-to on testing your app on JDK 8, JDK 9, and JDK 10 Early Access using Travis CI.

Read More…