Here are a few questions I've been thinking about:

• How should I express data or API?
• How should the data be represented in Java or Scala?
• How do I convert the data into wire formats such as JSON?
• How do I evolve the data without breaking binary compatibility?

limitation of case class

The sealed trait and case class is the idiomatic way to represent datatypes in Scala, but it's impossible to add fields in binary compatible way. Take for example a simple case class Greeting, and see how it would expand into a class and a companion object:

I gave a talk at ScalaSphere with Dale Wijnand (@dwijnand). The state of sbt 0.13, sbt server, and sbt 1.0 (ScalaSphere ver):

Gigahorse 0.2.0 is now released. The new change is that it abstracts over two backends. @alexdupre contributed migration from AHC 1.9 to AHC 2.0, which is based on Netty 4 in #12.

In addition, there's now an experimental Akka HTTP support that I added. #15

Please see Gigahorse docs for the details.

gigahorse-github 0.1.0 is released. This is a Gigahorse plugin for Github API v3.

Here’s a quick example of how to get repository info:

scala> import gigahorse._, gigahorse.github.Github, scala.concurrent._, duration._
 
scala> val client = Github.localConfigClient
client: gigahorse.github.LocalConfigClient = LocalConfigClient(OAuthClient(****, List(StringMediaType(application/json), GithubMediaType(Some(v3),None,Some(json)))))
 
scala> Gigahorse.withHttp { http =>
         val f = http.run(client(Github.repo("eed3si9n", "gigahorse-github")), Github.asRepo)
         Await.result(f, 2.minutes)
       }
res0: gigahorse.github.response.Repo = Repo(https://api.github.com/repos/eed3si9n/gigahorse-github, gigahorse-github, 64614221, User(https://api.github.com/users/eed3si9n, eed3si9n, 184683, Some(https://github.com/eed3si9n), Some(https://avatars.githubusercontent.com/u/184683?v=3), Some(), Some(User), Some(true), None, None), eed3si9n/gigahorse-github, Some(Gigahorse plugin for Github API v3),...

If you're interested in gigahorse-github itself, README contains the full documentation.

extending Gigahorse

I also wrote Extending Gigahorse page describing the overview of how to write a Gigahorse plugin, which is more or less the same as how one would write a Dispach plugin. As I wrote there, the JSON data binding is auto generated from a schema.

For me, gigahorse-github was as much a proof of concept for sbt-datatype as it was for Gigahorse. It did end up exposing minor bugs on all components along the stack, so it was a fruitful exercise.

Update: please use Gigahorse 0.1.1

Gigahorse 0.1.0 is now released. It is an HTTP client for Scala with Async Http Client underneath. Please see Gigahorse docs for the details. Here's an example snippet to get the feel of the library.

scala> import gigahorse._
scala> import scala.concurrent._, duration._
scala> Gigahorse.withHttp(Gigahorse.config) { http =>
         val r = Gigahorse.url("http://api.duckduckgo.com").get.
           addQueryString(
             "q" -> "1 + 1",
             "format" -> "json"
           )
         val f = http.run(r, Gigahorse.asString andThen {_.take(60)})
         Await.result(f, 120.seconds)
       }

There's a "pattern" that I've been thinking about, which arises in some situation while persisting/serializing objects.

To motivate this, consider the following case class:

scala> case class User(name: String, parents: List[User])
defined class User
 
scala> val alice = User("Alice", Nil)
alice: User = User(Alice,List())
 
scala> val bob = User("Bob", alice :: Nil)
bob: User = User(Bob,List(User(Alice,List())))
 
scala> val charles = User("Charles", bob :: Nil)
charles: User = User(Charles,List(User(Bob,List(User(Alice,List())))))
 
scala> val users = List(alice, bob, charles)
users: List[User] = List(User(Alice,List()), User(Bob,List(User(Alice,List()))),
  User(Charles,List(User(Bob,List(User(Alice,List()))))))

The important part is that it contains parents field, which contains a list of other users.
Now let's say you want to turn users list of users into JSON.

[{ "name": "Alice", "parents": [] },
{ "name": "Bob",
  "parents": [{ "name": "Alice", "parents": [] }] },
{ "name": "Charles",
  "parents": [{ "name": "Bob", "parents": [{ "name": "Alice", "parents": [] }] }] }]

This is part 3 on the topic of sjson-new. See also part 1 and part 2.

Within the sbt code base there are a few places where the persisted data is in the order of hundreds of megabytes that I suspect it becomes a performance bottleneck, especially on machines without an SSD drive.
Naturally, my first instinct was to start reading up on the encoding of Google Protocol Buffers to implement my own custom binary format.

microbenchmark using sbt-jmh

What I should've done first, is start benchmarking. Using @ktosopl (Konrad Malawski)'s sbt-jmh, setting up a microbenchmark is easy. All you have to do is pop that plugin into your build. and create a subproject that enables JmhPlugin.

Two months ago, I wrote about sjson-new. I was working on that again over the weekend, so here's the update.
In the earlier post, I've introduced the family tree of JSON libraries in Scala ecosystem, the notion of backend independent, typeclass based JSON codec library. I concluded that we need some easy way of defining a custom codec for it to be usable.

roll your own shapeless

In between the April post and the last weekend, there were flatMap(Oslo) 2016 and Scala Days New York 2016. Unfortunately I wasn't able to attend flatMap, but I was able to catch Daniel Spiewak's "Roll Your Own Shapeless" talk in New York. The full flatMap version is available on vimeo, so I recommend you check it out.

sbt internally uses HList for caching using sbinary:

implicit def mavenCacheToHL = (m: MavenCache) => m.name :*: m.rootFile.getAbsolutePath :*: HNil
implicit def mavenRToHL = (m: MavenRepository) => m.name :*: m.root :*: HNil
...

and I've been thinking something like an HList or Shapeless's LabelledGeneric would be a good intermediate datatype to represent JSON object, so Daniel's talk became the last push on my back.
In this post, I will introduce a special purpose HList called LList.

LList

sjson-new comes with a datatype called LList, which stands for labelled heterogeneous list.
List[A] that comes with the Standard Library can only store values of one type, namely A. Unlike the standard List[A], LList can store values of different types per cell, and it can also store a label per cell. Because of this reason, each LList has its own type. Here's how it looks in the REPL:

scala> import sjsonnew._, LList.:*:
import sjsonnew._
import LList.$colon$plus$colon
 
scala> import BasicJsonProtocol._
import BasicJsonProtocol._
 
scala> val x = ("name", "A") :*: ("value", 1) :*: LNil
x: sjsonnew.LList.:*:[String,sjsonnew.LList.:*:[Int,sjsonnew.LNil]] = (name, A) :*: (value, 1) :*: LNil
 
scala> val y: String :*: Int :*: LNil = x
y: sjsonnew.LList.:*:[String,sjsonnew.LList.:*:[Int,sjsonnew.LNil]] = (name, A) :*: (value, 1) :*: LNil

Can you find String and Int mentioned in that long type name of x? String :*: Int :*: LNil is a short form of writing that as demonstrated by y.

BasicJsonProtocol is able to convert all LList values into a JSON object.

I made a new Github organization called foundweekends for people who like coding in the weekends. If you want to join, or have project ideas ping me on twitter or come talk to us on Gitter.

As the starter, it will pick up maintenance of conscript, giter8, and pamflet from @n8han.

As a favorite weekend activity for the Scala programmers, I wrote my own JSON library called sjson-new.
sjson-new is a typeclass based JSON codec library, or wit for that Jawn. In other words, it aims to provide sjson-like codec facility in a backend independent way.

In terms of the codebase I based it off of spray-json, but conceptually it's close to Scala Pickling in the way it deals with data. Unlike Pickling, however, sjson-new-core is free of macros and runtime reflection beyond normal pattern matching.

Here's how to use with Json4s-AST:

libraryDependencies += "com.eed3si9n" %%  "sjson-new-json4s" % "0.1.0"

Here's how to use with Spray:

libraryDependencies += "com.eed3si9n" %%  "sjson-new-spray" % "0.1.0"