As of Scala 2.13.0-M5, it’s planned that scala.Seq will change from scala.collection.Seq to scala.collection.immutable.Seq. Scala 2.13 collections rework explains a bit about why it’s been non-immutable historically. Between the lines, I think it’s saying that we should celebrate that scala.Seq will now be immutable out of the box.
Defaulting to immutable sequence would be good for apps and fresh code. The situation is a bit more complicated for library authors.
3 h 28 min
We need to change the culture around tech conferences to improve the inclusion of women (and people from other backgrounds too!). For that, there needs to be clear signaling and communication about two basic issues
Red Mars by Kim Stanley Robinson
My rating: 4 of 5 stars
Red Mars is a science fiction classic written in 1992. This book is everything I wanted and more, starting with first hundred astronauts and cosmonauts migrating to Mars to build the infrastructure such that more people can migrate. By the middle of the book, thousands of people migrate to Mars. The book explores various dimensions the epic project, not just technological challenges, but psychological effects, personality differences, and political interests by superpowers. The geography of Mars is written in vivid, majestic details.
I’ve implemented “super shell” feature for sbt over the weekend. The idea is to take over the bottom n lines of the termnial, and display the current tasks in progress.
Logs are useful in many situations, and sometimes it’s the only viable tool to find out what’s going on. But on a console app like sbt, using logs to tell the build user what’s going on doesn’t always work.
Compile, or compile not. There’s no warning. Two of my favorite Scala compiler flags lately are "-Xlint" and "-Xfatal-warnings".
Here is an example setting that can be used with subprojects:
ThisBuild / organization := "com.example"
ThisBuild / version := "0.1.0-SNAPSHOT"
ThisBuild / scalaVersion := "2.12.6"
lazy val commonSettings = List(
scalacOptions ++= Seq(
"-encoding", "utf8",
"-deprecation",
"-unchecked",
"-Xlint",
"-feature",
"-language:existentials",
"-language:experimental.macros",
"-language:higherKinds",
"-language:implicitConversions",
"-Ypartial-unification",
"-Yrangepos",
),
scalacOptions ++= (scalaVersion.value match {
case VersionNumber(Seq(2, 12, _*), _, _) =>
List("-Xfatal-warnings")
case _ => Nil
}),
Compile / console / scalacOptions --= Seq("-deprecation", "-Xfatal-warnings", "-Xlint")
)
lazy val foo = (project in file("foo"))
.settings(
commonSettings,
name := "foo",
)
Read More…
Working with GitHub and pull requests a lot, I end up accumulating stale branches that are no longer needed. In this post, we will look at how to clean the stale local branches.
There are mainly two strategies:
Erik Aybar’s Git Tip: Deleting Old Local Branches takes the second approach.
I’ve been thinking about rich console applications, the kind of apps that can display things graphically, not just appending lines at the end. Here are some info, enough parts to be able to write Tetris.
To display some text at an arbitrary location on a termial screen, we first need to understand what a terminal actually is. In the middle of 1960s, companies started selling minicomputers such as PDP-8, and later PDP-11 and VAX-11. These were of a size of a refrigerator, purchased by “computer labs”, and ran operating systems like RT-11 and the original UNIX system that supported up many simultaneous users (12 ~ hundreds?). The users connected to a minicomputer using a physical terminal that looks like a monochrome screen and a keyboard. The classic terminal is VT100 that was introduced in 1978 by DEC.
Whether you want to try using OpenJDK 11-ea, GraalVM, Eclipse OpenJ9, or you are stuck needing to build using OpenJDK 6, jabba has got it all. jabba is a cross-platform Java version manager written by Stanley Shyiko (@shyiko).
Here’s how we can use jabba on Travis CI to cross build using AdoptOpenJDK 8 and 11:
sudo: false
dist: trusty
group: stable
language: scala
scala:
- 2.12.7
env:
global:
- JABBA_HOME=/home/travis/.jabba
matrix:
include:
- env:
- TRAVIS_JDK=adopt@1.8.192-12
- env:
- TRAVIS_JDK=adopt@1.11.0-1
before_install:
- curl -sL https://raw.githubusercontent.com/shyiko/jabba/0.11.0/install.sh | bash && . ~/.jabba/jabba.sh
install:
- $JABBA_HOME/bin/jabba install $TRAVIS_JDK
- unset _JAVA_OPTIONS
- export JAVA_HOME="$JABBA_HOME/jdk/$TRAVIS_JDK" && export PATH="$JAVA_HOME/bin:$PATH" && java -Xmx32m -version
script: sbt -Dfile.encoding=UTF8 -J-XX:ReservedCodeCacheSize=256M ++$TRAVIS_SCALA_VERSION! test
before_cache:
- find $HOME/.ivy2 -name "ivydata-*.properties" -delete
- find $HOME/.sbt -name "*.lock" -delete
cache:
directories:
- $HOME/.ivy2/cache
- $HOME/.sbt/boot
- $HOME/.jabba/jdk
Read More…
Last week I wrote about using source dependencies with sbt-sriracha for testing purpose. This week we’ll look into using Expecty to do power assert.
Power assert (or power assertion) is a variant of assert(...) function that that prints out detailed error message automatically. It was originally implemented by Peter Niederwieser (@pniederw) for Spock, and in 2009 it was merged into Groovy 1.7. Power assert has spread to Ruby, JavaScript, Rust, etc.
Source dependencies is one of features that existed in sbt since ever, but hasn’t been documented well.
Here’s how to declare source dependency to the latest commit for scopt commandline option parsing library.
lazy val scoptJVMRef = ProjectRef(uri("git://github.com/scopt/scopt.git#c744bc48393e21092795059aa925fe50729fe62b"), "scoptJVM")
ThisBuild / organization := "com.example"
ThisBuild / scalaVersion := "2.12.2"
lazy val root = (project in file("."))
.dependsOn(scoptJVMRef)
.settings(
name := "Hello world"
)
Read More…
Yesterday I wrote about cross JVM testing using Travis CI.
Here’s how we can test Scala apps on macOS using Travis CI. This is adapted from Lars and Muuki’s method: Testing Scala programs with Travis CI on OS X
dist: trusty
language: scala
matrix:
include:
## build using JDK 8, test using JDK 8
- script:
- sbt universal:packageBin
- cd citest && ./test.sh
jdk: oraclejdk8
## build using JDK 8, test using JDK 8, on macOS
- script:
- sbt universal:packageBin
- cd citest && ./test.sh
## https://github.com/travis-ci/travis-ci/issues/2316
language: java
os: osx
osx_image: xcode9.2
## build using JDK 8, test using JDK 9
- script:
- sbt universal:packageBin
- jdk_switcher use oraclejdk9
- cd citest && ./test.sh
jdk: oraclejdk8
## build using JDK 8, test using JDK 10
- script:
- sbt universal:packageBin
- citest/install-jdk10.sh
- cd citest && ./test.sh
jdk: oraclejdk8
scala:
- 2.10.7
before_install:
# https://github.com/travis-ci/travis-ci/issues/8408
- unset _JAVA_OPTIONS
- if [[ "$TRAVIS_OS_NAME" = "osx" ]]; then
brew update;
brew install sbt;
fi
cache:
directories:
- $HOME/.ivy2/cache
- $HOME/.sbt/boot
before_cache:
- find $HOME/.ivy2 -name "ivydata-*.properties" -delete
- find $HOME/.sbt -name "*.lock" -delete
Read More…
Oracle is moving to ship non-LTS JDK every 6 months, and LTS JDK every 3 years. Also it’s converging to OpenJDK. In this scheme, JDK 9 will be EOL in March 2018; JDK 10 will come out in March 2018, and EOL in September 2018; and LTS JDK 11 that replaces JDK 8 in September 2018 will stay with us until 2021.
As we will see quick succession of JDKs in the upcoming months, here’s a how-to on testing your app on JDK 8, JDK 9, and JDK 10 Early Access using Travis CI.
During the SIP-27 trailing commas discussion, one of the thoughts that came to my mind was unifiying some of the commas with semicolons, and take advantage of the semicolon inference.
This doesn’t actually work. @Ichoran kindly pointed out an example:
As tweeted, I made some sbt stickers.
Wrote herding cats: day 17 featuring initial and terminal objects, product, duality, and coproduct.
This holiday break, I somehow got into binge watching Coursera’s Stanford Machine Learning course taught by Andrew Ng. I remember machine learning to be really math heavy, but I found this one more accessible.
Here are some notes for my own use. (I am removing all the fun examples, and making it dry, so if you’re interested in machine learning, you should check out the course or its official notes.)
In this post I am going to discuss an old new problem of encoding file path as Uniform Resource Identifier (URI) reference.
As of 2017, the authoritative source of information is RFC 8089 - The “file” URI Scheme written by Matthew Kerwin.
RFC 8089 The "file" URI Scheme https://tools.ietf.org/html/rfc8089 Wow, it actually happened.
— Matthew Kerwin (@phluid61) February 18, 2017
Future readers might also want to search for “file URI scheme RFC”, and find the latest version. If you’re a programmer, read the RFC. This post is to raise the awareness of the some of the issues around file to URI encoding, but it’s not a substitution.
On Tech Hub blog I demonstrated how to use sbt server from VS Code to display compiler errors from a running sbt session. In this post, I’ll show how to do that for Sublime Text 3 in this post.
First, add tomv564/LSP plugin to Sublime Text 3.
cd ~/Library/Application\ Support/Sublime\ Text\ 3/Packagesgit clone https://github.com/tomv564/LSP.gitNext, download sbt-server-stdio.js and save it to ~/bin/ or somewhere you keep scripts. sbt server by default uses Unix domain sockets on POSIX systems and named pipe on Windows, but editors seem to expect stdio. The script is a Node script that’s included as our VS Code extension that discovers the socket, and fronts it with stdio.
Over the weekend I assembled an Ergodox.
It’s been a month since sbt 1.0 shipped, and I can finally sit back and think about sbt server again. Using my weekends time, I started hacking on an implementation of Scala language server on top of sbt server.
A language server is a program that can provide language service to editors like Visual Studio Code, Eclipse Che, and Sublime Text 3 via Language Server Protocol. A typical operation might be textDocument/didOpen, which tells the server that a source file was opened in the editor.
In this post, I’d like to introduce a version scheme that I call Persistent Versioning. Most of the ideas presented in this post are not new or my own. Let me know if there’s already a name for it.
In 2015, Jake Wharton (@JakeWharton) wrote a blog post titled Java Interoperability Policy for Major Version Updates:
A new policy from @jessewilson and I for the libraries we work on to ensure major version updates are interoperable: https://t.co/zKqYRwrXmq
GitHub Pages is a convenient place to host OSS project docs. This post explains how to use Travis CI to deploy your docs automatically on a pull request merge.
Make a directory outside of your project first.
Pick a key name deploy_yourproject_rsa, so you can distinguish it from other keys.
$ mkdir keys
$ cd keys
$ ssh-keygen -t rsa -b 4096 -C "yours@example.com"
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/xxx/.ssh/id_rsa): deploy_website_rsa
Enter passphrase (empty for no passphrase):
Read More…
In the last post that I wrote about Atreus build, I noted that there’s an issue of keyboard positioning:
Even if I can overcome the layout and memorize the various symbol locations, there’s the issue of the placement. If I place the keyboard in between me and the laptop the screen becomes too far.
I solved this issue by making a tray for Atreus that I can position it on top of the MacBook Pro keyboard.
Last night I finished making my Atreus keyboard from a DYI kit that I got a while back. Here are some of the details:
The kit comes with almost everything you need to assemble the Arteus keyboard. You need lacquer, a soldering iron, solder, and wire cutters.
Gigahorse 0.3.0 is now released. See documentation on what it is.
0.3.0 adds Square OkHttp support. Gigahorse-OkHttp is availble for Scala 2.10, 2.11, and 2.12.
According to the JavaDoc you actually don’t have to close the OkHttpClient instance.
scala> import gigahorse._, support.okhttp.Gigahorse
import gigahorse._
import support.okhttp.Gigahorse
scala> import scala.concurrent._, duration._
import scala.concurrent._
import duration._
scala> val http = Gigahorse.http(Gigahorse.config) // don't have to close
http: gigahorse.HttpClient = gigahorse.support.okhttp.OkhClient@23b48158
I’ve been asked by a few people on downloading JARs, and then running them from an sbt plugin. Most recently, Shane Delmore (@shanedelmore) asked me about this at nescala in Brooklyn.
During an unconference session I hacked together a demo, and I continued some more after I came home.
sbt-sidedish is a toolkit for plugin authors to download and run an app on the side from a plugin. It on its own does not define any plugins.
Wrote herding cats day 16.
Here are a few questions I’ve been thinking about:
The sealed trait and case class is the idiomatic way to represent datatypes in Scala, but it’s impossible to add fields in binary compatible way. Take for example a simple case class Greeting, and see how it would expand into a class and a companion object:
Gigahorse 0.2.0 is now released. The new change is that it abstracts over two backends. @alexdupre contributed migration from AHC 1.9 to AHC 2.0, which is based on Netty 4 in #12.
In addition, there’s now an experimental Akka HTTP support that I added. #15
Please see Gigahorse docs for the details.
gigahorse-github 0.1.0 is released. This is a Gigahorse plugin for Github API v3.
Update: please use Gigahorse 0.1.1
Gigahorse 0.1.0 is now released. It is an HTTP client for Scala with Async Http Client underneath. Please see Gigahorse docs for the details. Here’s an example snippet to get the feel of the library.
scala> import gigahorse._
scala> import scala.concurrent._, duration._
scala> Gigahorse.withHttp(Gigahorse.config) { http =>
val r = Gigahorse.url("http://api.duckduckgo.com").get.
addQueryString(
"q" -> "1 + 1",
"format" -> "json"
)
val f = http.run(r, Gigahorse.asString andThen {_.take(60)})
Await.result(f, 120.seconds)
}
There’s a “pattern” that I’ve been thinking about, which arises in some situation while persisting/serializing objects.
To motivate this, consider the following case class:
scala> case class User(name: String, parents: List[User])
defined class User
scala> val alice = User("Alice", Nil)
alice: User = User(Alice,List())
scala> val bob = User("Bob", alice :: Nil)
bob: User = User(Bob,List(User(Alice,List())))
scala> val charles = User("Charles", bob :: Nil)
charles: User = User(Charles,List(User(Bob,List(User(Alice,List())))))
scala> val users = List(alice, bob, charles)
users: List[User] = List(User(Alice,List()), User(Bob,List(User(Alice,List()))),
User(Charles,List(User(Bob,List(User(Alice,List()))))))
Read More…
This is part 3 on the topic of sjson-new. See also part 1 and part 2.
Within the sbt code base there are a few places where the persisted data is in the order of hundreds of megabytes that I suspect it becomes a performance bottleneck, especially on machines without an SSD drive. Naturally, my first instinct was to start reading up on the encoding of Google Protocol Buffers to implement my own custom binary format.
Two months ago, I wrote about sjson-new. I was working on that again over the weekend, so here’s the update. In the earlier post, I’ve introduced the family tree of JSON libraries in Scala ecosystem, the notion of backend independent, typeclass based JSON codec library. I concluded that we need some easy way of defining a custom codec for it to be usable.
In between the April post and the last weekend, there were flatMap(Oslo) 2016 and Scala Days New York 2016. Unfortunately I wasn’t able to attend flatMap, but I was able to catch Daniel Spiewak’s “Roll Your Own Shapeless” talk in New York. The full flatMap version is available on vimeo, so I recommend you check it out.
I made a new Github organization called foundweekends for people who like coding in the weekends. If you want to join, or have project ideas ping me on twitter or come talk to us on Gitter.
As the starter, it will pick up maintenance of conscript, giter8, and pamflet from @n8han.
One of the fun way of thinking about software projects is literary analysis. Instead of the actual source code, think of who wrote it when and why (what problem does it solve), and how it’s written (what influenced it). Within the Scala ecosystem, not too many genre are as rich as the JSON libraries.
In December 2008, the first edition of Programming in Scala came out, which used JSON as an example in the context of parser combinator, and showed that JSON parser can be written in 10 lines of code:
This is a continuation from the sbt 1.0 roadmap that I wrote recently. In this post, I’m going to introduce a new implementation of sbt server. Please post on sbt-dev mailing list for feedback.
The motivation for sbt server is better IDE integration.
A build is a giant, mutable, shared, state, device. It’s called disk! The build works with disk. You cannot get away from disk.
– Josh Suereth in The road to sbt 1.0 is paved with server
The disk on your machine is fundamentally a stateful thing, and sbt can execute the tasks in parallel only because it has the full control of the effects. Any time you are running both sbt and an IDE, or you’re running multiple instances of sbt against the same build, sbt cannot guarantee the state of the build.
There’s been some discussions around sbt 1.0 lately, so here is a writeup to discuss it. This document is intended to be a mid-term mission statement. A refocus to get something out. Please post on sbt-dev mailing list for feedback.
I don’t have a good idea on the timing of when sbt 1.0 will ship. The biggest feature of sbt 1.0 is code reorganization, which is is already in progress: http://www.scala-sbt.org/0.13/docs/Modularization.html
For me (and for many of the 27 organizers, I imagine) ScalaMatsuri is a lifestyle. It’s true that there was a successful two-day conference in Tokyo with 550 participants. But for us the organizers, the preparation has been going on since February 28th, for 11 months. Despite the fact that my contribution was small, planning ScalaMatsuri 2016 was by far the most amount of involvement I’ve committed to. Through the course of planning months, there were many discussions over Slack, Hangouts, and occasionally even face-to-face. The fun part was coming up with the ideas together, and seeing them materialize. Sometimes, I was the one coming up with radical ideas that were being executed by someone else, while other times, it was the opposite case and I was getting my hands dirty.
For a flexible language like Scala, it’s useful to think of subset of the programming language, like your own personal Good Parts, and opinionated style guides.
To try -Yno-lub, you can drop in the following sbt plugin to project/ynolub.sbt:
addSbtPlugin("com.eed3si9n" % "sbt-ynolub" % "0.2.0")
When Scala’s type inferencer finds type A and type B to unify, it tries to calculate the lub (least upper bounds) of two types with regards to <:<. This process is sometimes called lubbing. Here are some of the examples:
Check out my new series herding cats. (I’m writing it using Pamflet from the get go)
This is a log of me going through Cats, a functional programming library for Scala that is currently experimental and under active development.
I gave a talk at Scala Days 2015 San Francisco with Josh Suereth (@jsuereth).
I’ve been switching between Mac and Ubuntu, and between Java 6 and 7 lately. This is a memo of how to switch Java versions on both Mac and Ubuntu.
Update: Yoshida-san told me about this thing called jEnv, which does all this.
Here’s one way of loading different shell files depending on the OS:
## basic
[ -f $HOME/dotfiles/zshrc.basic ] && source $HOME/dotfiles/zshrc.basic
## aliases
[ -f $HOME/dotfiles/zshrc.alias ] && source $HOME/dotfiles/zshrc.alias
case "${OSTYPE}" in
# MacOSX
darwin*)
[ -f $HOME/dotfiles/zshrc.osx ] && source $HOME/dotfiles/zshrc.osx
;;
# Linux
linux*)
[ -f $HOME/dotfiles/zshrc.linux ] && source $HOME/dotfiles/zshrc.linux
;;
esac
## color
[ -f $HOME/dotfiles/zshrc.color ] && source $HOME/dotfiles/zshrc.color
Read More…
On my way back from Uppsala, my mind wandered to a conversation I had with a collegue about the intuition of monads, which I pretty much butchered at the time. As I was mulling this over, it dawned on me.
The above is a fractal called Sierpinski triangle, the only fractal I can remember to draw. Fractals are self-similar structure like the above triangle, in which the parts are similar to the whole (in this case exactly half the scale as parent triangle).
Monads are fractals. Given a monadic data structure, its values can be composed to form another value of the data structure. This is why it’s useful to programming, and this is why it occurrs in many situations.
Let’s look at some examples:
scala> List(List(1), List(2, 3), List(4))
res0: List[List[Int]] = List(List(1), List(2, 3), List(4))
Two days of #ScalaMatsuri ended as a huge success. But for the next year, I’m leaving ourselves a few homeworks to work on. As the title suggests, the next goal that we should aim for is universal access. In Scala language, universal access principle indicates the fact that both methods and fields can be accessed interchangeably from outside.
For a conference, I mean universal access to mean being more inclusive to various groups of people:
This year was the second Scala conference in Japan. We’ve changed the name to ScalaMatsuri, which means Scala festival in Japanese. 300 tickets sold out. With invited guests and free tickets for sponsors, there may have been even more people. The venue was at CyberAgent, which runs blog service and online ad services.
Day 1 kicked off with Martin’s (@ordersky) ‘Evolution of Scala.’ Many people were looking forward to see Martin, so I think it was full house from the get go. During all sessions that I attended, I was busy typing live text translation using my closed-captioning both from English to Japanese, and from Japanese to English along with other members @cbirchall, @cdepillabout, @okapies, and @oe_uia.