I’m hacking on a small project called sudori, an experimental sbt. The initial goal is to port the macro to Scala 3. It’s an exercise to take the macro apart and see if we can build it from the ground up. This an advanced area of Scala 2 and 3, and I’m finding my way around by trial and error. This is part 3.
It’s been a while since I wrote part 2, but in between I’ve written intro to Scala 3 macros, which is sort of a sudori prequel.
Starlark is a dialect of Python, originally designed as a configuration language for the Bazel build tool. Currently there are implementations in Go, Java, and Rust. As far as I know, the main Java implementation of Starlark has only been available as Bazel’s source repo on GitHub.
Since it would be convenient to have a binary distribution, I’ve forked the repo, and published it as "com.eed3si9n.starlark" % "starlark" % "4.2.1" (com.eed3si9n.starlark:starlark:4.2.1) on Maven Central. The code is the same as Bazel 4.2.1.
Here’s a quick tutorial of how to test your project on JDK 17 using Ólaf’s olafurpg/setup-scala. As the starting point we’ll use the following setup, which is documented in Setting up GitHub Actions with sbt:
name: CI
on:
pull_request:
push:
jobs:
test:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
java: 11
jobtype: 1
- os: ubuntu-latest
java: 11
jobtype: 2
- os: ubuntu-latest
java: 11
jobtype: 3
runs-on: ${{ matrix.os }}
steps:
- name: Checkout
uses: actions/checkout@v1
- name: Setup
uses: olafurpg/setup-scala@v13
with:
java-version: "adopt@1.${{ matrix.java }}"
- name: Build and test
run: |
case ${{ matrix.jobtype }} in
1)
sbt -v "mimaReportBinaryIssues; scalafmtCheckAll; +test;"
;;
2)
sbt -v "scripted actions/*"
;;
3)
sbt -v "dependency-management/*"
;;
*)
echo unknown jobtype
exit 1
esac
shell: bash
Read More…
Macro is a fun and powerful tool, but overuse of the macro could cause harm as well. Please enjoy macros responsibly.
What is macro? A common explanation given is that a macro is a program that is able to take code as an input and output code. While it’s true, it might not immediately make sense since Scala programmers are often familiar with higher-order functions like (map {...}) and by-name parameter, which on the surface it might seem like it is passing a block of code around.
Jar Jar Abrams 1.8.0 and sbt-assembly 1.1.0 are released.
Jar Jar Abrams is an experimental extension to Jar Jar Links, intended to shade Scala libraries. Thus far we have been using Pants team’s fork of Jar Jar Links, but now that it’s been abandaned, Eric Peters has in-sourced it to jarjar-abrams repo so we can patch it.
Our jarjar fork is released under com.eed3si9n.jarjar organization name and package name.
I’m hacking on a small project called sudori, an experimental sbt. The initial goal is to port the macro to Scala 3. It’s an exercise to take the macro apart and see if we can build it from the ground up. This an advanced area of Scala 2 and 3, and I’m finding my way around by trial and error. This is part 2.
Reference:
When we think of the build.sbt macro, the first thing that comes to our mind is the Applicative do macro that it implements using .value even though some may not use those terms exactly. The main driver for this imperative-to-functional is in the companion object for an oddly named Instance class:
I’m hacking on a small project called sudori, an experimental sbt. The initial goal is to port the macro to Scala 3. It’s an exercise to take the macro apart and see if we can build it from the ground up. This an advanced area of Scala 2 and 3, and I’m finding my way around by trial and error.
Reference:
I think I’ve identified a basic part called Convert, which doesn’t really depend on anything.
I’m happy to announce sbt 1.5.5 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.5.5
buildTarget/resources support for BSP #6552 by @samuelClarencTeadsbuild.sbt support for BSP import #6553 by @retronymNoClassDefFoundError when launching sbt 1.4.0 - 1.4.2 launcher#98 by @eed3si9n_3 lm#383 by @eed3si9nI’m happy to announce sbt 1.5.4 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.5.4
Download the official sbt runner + launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.5.4
This mechanism allows that sbt 1.5.4 is used only for the builds that you want.
In June of 2011, I started working on sbt-assembly for sbt 0.10, based on Coda Hale’s assembly-sbt from sbt 0.7, which in turn was probably inspired by maven-assembly-plugin. After ten years, I’m going to call this one 1.0.0. sbt-assembly 1.0.0 is published to Maven Central.
I’m happy to announce sbt 1.5.3 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.5.3
Download the official sbt runner + launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.5.3
This mechanism allows that sbt 1.5.3 is used only for the builds that you want.
| syntax | Scala | Python |
|---|---|---|
| immutable variable | val x = 1 |
Starlark:REV = "1.1.0" |
| lazy variable | lazy val x = 1 |
n/a |
| mutable variable | var x = 1 |
in function:x = 1 |
| if expression | if (x > 1) "a" else "b" |
"a" if x > 1 else "b" |
| ———————- | —————————- | —————————- |
| function | def add3(x: Int): Int = x + 3 |
def add3(x): return x + 3 |
| anonymous function | _ * 2 |
not in Starlark:lambda x: x * 2 |
| ———————- | —————————- | —————————- |
| List | val xs = List(1, 2, 3, 4) |
xs = [1, 2, 3, 4] |
| size | xs.size |
len(xs) |
| empty test | xs.isEmpty |
not xs |
| head | xs.head |
xs[0] |
| tail | // List(2, 3, 4)xs.tail |
# [2, 3, 4]xs[1:] |
| take | // List(1, 2)xs.take(2) |
# [1, 2]xs[:2] |
| drop | // List(3, 4)xs.drop(2) |
# [3, 4]xs[2:] |
| drop right | // List(1, 2, 3)xs.dropRight(1) |
# [1, 2, 3]xs[:-1] |
| nth element | xs(2) |
xs[2] |
| map | xs.map(_ * 2)for { x <- xs} yield x * 2 |
map(lambda x: x * 2, xs)[x * 2 for x in xs] |
| filter | xs.filter(_ % 2 == 0)for { x <- xs if x % 2 == 0} yield x |
filter(lambda x: not x % 2, xs)[x for x in xs if not x % 2 ] |
| fold from left | // "a1234"xs.foldLeft("a") { _ + _ } |
from functools import reduce# "a1234"reduce(lambda a,x: a + str(x), xs, "a") |
| membership | xs.contains(3) |
3 in xs |
| ———————- | —————————- | —————————- |
| String | val s = "hello" |
s = "hello" |
| variable interpolation | val count = 3s"$count items" |
not in Starlark:count = 3f"{count} items" |
| split | // Array(1.2.3, M1)"1.2.3-M1".split("-") |
# ['1.2.3', 'M1']"1.2.3-M1".split("-") |
| substring test | s.contains("el") |
"el" in s |
| ———————- | —————————- | —————————- |
| Map | val d = Map("a" -> 1, “b” -> 2) |
d = { "a": 1, "b": 2 } |
I’m picking up Python and its Bazel dialect Skylark lately. On the other hand, I’m familiar with Scala and its sbt dialect. Often I know exactly what I want to express, and I am fairly certain Python has the equivalent concept as Scala, but just don’t remember the exact incantation. For people starting Scala, maybe they could use this table in reverse.
There’s a long-standing bug that sbt maintainers have known for a while, which is that when sbt plugin is published to a Maven repository, the POM file sbt generates is not valid. From a mailing list thread titled [0.12] plan for instance, Mark McBride reported it in 2012:
On the maven note, the poms generated for plugins aren’t actually valid. Trying to upload them to artifactory without disabling pom consistency checks fails :/
I’m happy to announce sbt 1.5.2 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.5.2
Download the official sbt runner + launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.5.2
This mechanism allows that sbt 1.5.2 is used only for the builds that you want.
I’m happy to announce sbt 1.5.1 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.5.1. This post will also report the Bintray to JFrog Artifactory migration.
First and foremost, I would like to thank JFrog for their continued support of sbt project and the Scala ecosystem.
As sbt was taking off in the number of contributors and plugins, we had a Bintray-shaped problem. We wanted individuals to create Ivy-layout repository, publish sbt plugins, but somehow aggregate the resolution to them. Having Github sbt organization allowed fluid ownership of plugin sources, but distributing the binary files were challenge as sbt version was churning. We adopted Bintray in 2014 and it provided the distribution mechanism during our growth years. In addition, we used Bintray to host Debian and RPM installers for sbt, paid for by Lightbend.
Wrote herding cats: day 19 featuring FunctionK, or Rúnar’s encoding of rank-2 polymorphic function, and Resource datatype, which he envisioned rank-N polymorphism would unlock back in 2010.
Hi everyone. On behalf of the sbt project, I am happy to announce sbt 1.5.0. This is the fifth feature release of sbt 1.x, a binary compatible release focusing on new features. sbt 1.x is released under Semantic Versioning, and the plugins are expected to work throughout the 1.x series.
The headline features of sbt 1.5.0 are:
Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/tag/v1.5.0. This installer includes the new Coursier-based launcher.
Hi everyone. On behalf of the sbt project, I am happy to announce sbt 1.5.0-RC2. This is the fifth feature release of sbt 1.x, a binary compatible release focusing on new features. sbt 1.x is released under Semantic Versioning, and the plugins are expected to work throughout the 1.x series.
The headline features of sbt 1.5.0 are:
I’m happy to announce sbt 1.4.9 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.9
Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.4.9
This mechanism allows that sbt 1.4.9 is used only for the builds that you want.
I’m happy to announce sbt 1.4.8 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.8
Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.4.8
This mechanism allows that sbt 1.4.8 is used only for the builds that you want.
In sbt 1.1.0 I implemented unified slash syntax for sbt. Today I sent a pull request to deprecate the old sbt 0.13 shell syntax #6309.
Naturally, the topic of deprecating old syntax for build.sbt also came up.
will you also deprecate `scalacOptions in (Compile, console)` in *.sbt and *.scala files? I hope so
— Seth Tisue (@SethTisue) February 16, 2021
This is because “unified” slash syntax is called so because it unifies the shell syntax and the build syntax together. Thus, it makes sense to deprecate the old build.sbt syntax that uses in like skip in publish or scalacOptions in (Compile, console), if we’re deprecating the old shell syntax.
bisect.sh can save a lot of time by using the pre-build compiler artifacts on the Scala CI Artifactory.
I’m happy to announce sbt 1.4.7 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.7
Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.4.7
This mechanism allows that sbt 1.4.7 is used only for the builds that you want.
I’m happy to announce sbt 1.4.6 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.6
Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.4.6
This mechanism allows that sbt 1.4.6 is used only for the builds that you want.
I want to tell sbt “this specific version breaks binary compatibility, so don’t resolve it via eviction, fail the build instead.” How do I do this? Complete answers only, I’m done trying to figure it out by following clues.
I wrote a small sbt plugin sbt-strict-update to do this.
Add this to project/plugins.sbt:
addSbtPlugin("com.eed3si9n" % "sbt-strict-update" % "0.1.0")
Read More…
I’m happy to announce sbt 1.4.5 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.5
Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.4.5
This mechanism allows that sbt 1.4.5 is used only for the builds that you want.
4h
This is a GitHub Actions version of auto publish sbt plugin from Travis CI.
In this post, we’ll try to automate the release of an sbt plugin using Ólaf’s olafurpg/sbt-ci-release. The README of sbt-ci-release covers the use case for a library published to Sonatype OSS. Read it thoroughly since this post will skip over the details that do not change for publishing sbt plugins.
Automated release in general is a best practice, but there’s one benefit specifically for sbt plugin releases. Using this setup allows multiple people to share the authorization to release an sbt plugin without adding them to Bintray sbt organization. This is useful for plugins maintained at work.
This post was first published in December 2018 together with 4.0.0-RC2. It’s updated to reflect the changes made in November 2020 for 4.0.0.
You can skip to the readme, if you’re in a hurry.
To try new scopt 4.0.0:
libraryDependencies += "com.github.scopt" %% "scopt" % "4.0.0"
scopt 4.0.0 is cross published for the following build matrix:
It’s a strange time we live in. We can’t just meet up and catch up and talk about coding. This also opens an opportunity to think more virtually about the idea of meetups.
I want to start Weehawken-Lang, a virtual meetup group about programming languages and tooling design (compilers, interpreters, build tools etc). It aims to be a casual place where people with different language backgrounds can exchange ideas about programming languages.
I’m happy to announce sbt 1.4.4 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.4
Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.4.4
This mechanism allows that sbt 1.4.4 is used only for the builds that you want.
I’m happy to announce sbt 1.4.3 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.3
Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/. This installer includes the sbtn binary.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.4.3
This mechanism allows that sbt 1.4.3 is used only for the builds that you want.
I’m happy to announce sbt 1.4.2 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.2
Download the official sbt launcher from SDKMAN or download from https://github.com/sbt/sbt/releases/. This installer includes the sbtn binary.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.4.2
This mechanism allows that sbt 1.4.2 is used only for the builds that you want.
The feature in sbt and Zinc 1.4.x that I spent most amount of time and energy probably is the virtualization of file, and lifting out timestamps. Combined together, we can liberate the Zinc state from machine-specificity and time, and become the foundation we lay towards building incremental remote caching for Scala. I blogged about this in cached compilation for sbt. This is part 2.
Now that sbt 1.4.x is out, there is a growing interest in this feature among people who want to try this out.
I’m happy to announce sbt 1.4.1 patch release is available. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.4.1
Download the official sbt launcher from SDKMAN or download from https://www.scala-sbt.org/download.html. This installer includes the sbtn binary.
In addition, the sbt version used for your build is upgraded by putting the following in project/build.properties:
sbt.version=1.4.1
This mechanism allows that sbt 1.4.1 is used only for the builds that you want.
Here’s a report of running a virtual hackathon at ScalaMatsuri Day 2 Unconference. Someone proposed it for the Unconference, and I volunteered to be a facilitator on the day, so I went in without preparation. I booked the time originally for 4h (noon - 4pm JST, 11pm - 3am EDT) but it was successful so it got extended after some coffee break.
One thing I emphasize is The Law of Two Feet:
I gave a talk at ScalaMatsuri on ‘Equality in Scala’
Hi everyone. On behalf of the sbt project, I am happy to announce sbt 1.4.0. This is the fourth feature release of sbt 1.x, a binary compatible release focusing on new features. sbt 1.x is released under Semantic Versioning, and the plugins are expected to work throughout the 1.x series.
The headline features of sbt 1.4.0 are:
ThisBuild / versionScheme to take the guessing out of eviction warningHi everyone. On behalf of the sbt project, I am happy to announce sbt 1.4.0-RC2. This is the fourth feature release of sbt 1.x, a binary compatible release focusing on new features. sbt 1.x is released under Semantic Versioning, and the plugins are expected to work throughout the 1.x series.
The headline features of sbt 1.4.0 are:
ThisBuild / versionScheme to take the guessing out of eviction warning
4h 29m
This is part 4 of the post about sbt-projectmatrix, an experimental plugin that I’ve been working to improve the cross building in sbt. Here’s part 1, part 2, and part 3. I’ve just released 0.6.0.
After adding sbt-projectmatrix to your build, here’s how you can set up a matrix with two Scala versions.
ThisBuild / organization := "com.example"
ThisBuild / scalaVersion := "2.12.12"
ThisBuild / version := "0.1.0-SNAPSHOT"
lazy val core = (projectMatrix in file("core"))
.settings(
name := "core"
)
.jvmPlatform(scalaVersions = Seq("2.12.12", "2.13.3"))
Read More…
I’m excited to announce that I’m joining Twitter’s Build Team to work on the next generation of efficient build systems supporting thousands of Twitter developers worldwide. Today’s my first day.
This is the team that developed monorepo build tool Pants, and is transitioning to migrate the flock to Bazel. This presented a unique opportunity for me to work with a team of people passionate about developer experience and productivity, and I’m looking forward to getting to know the team, and learning the new challenges.
I’m happy to announce sbt-buildinfo 0.10.0. sbt-buildinfo is a small sbt plugin to generate BuildInfo object from your build definitions.
Since the last feature release was in 2018, there have been some pending contributions. I think the important thing is that it compiles with -Xlint and -Xfatal-warnings on both Scala 2.13.3 and 2.12.12.
In this post, we’ll try to automate the release of an sbt plugin using Ólafur’s olafurpg/sbt-ci-release. The README of sbt-ci-release covers the use case for a library published to Sonatype OSS. Read it thoroughly since this post will skip over the details that do not change for publishing sbt plugins.
Automated release in general is a best practice, but there’s one benefit specifically for sbt plugin releases. Using this setup allows multiple people to share the authorization to release an sbt plugin without adding them to Bintray sbt organization. This is useful for plugins maintained at work.
Spotify: https://open.spotify.com/playlist/5BkvksBTbrngqXxhF6wUAc?si=Kj8TrnhRQICDTgadvYYt2w YouTube: https://www.youtube.com/playlist?list=PLSUh6oJ5ZotXfMWZPzm6K_L0CRPgFhxGh 2h 4m
I’m happy to announce sbt 1.3.13 patch release. Full release note is here - https://github.com/sbt/sbt/releases/tag/v1.3.13.
Special thanks to Scala Center. It takes time to review bug reports, pull requests, make sure contributions land to the right places, and Scala Center sponsored me to do maintainer tasks for sbt during June.