scala

warning: Creating default object from empty value in /opt/bitnami/apps/portal/htdocs/modules/taxonomy/taxonomy.pages.inc on line 33.

herding cats: day 17

in

Wrote herding cats: day 17 featuring initial and terminal objects, product, duality, and coproduct.

encoding file path as URI reference

in

In this post I am going to discuss an old new problem of encoding file path as Uniform Resource Identifier (URI) reference.

As of 2017, the authoritative source of information is RFC 8089 - The "file" URI Scheme written by Matthew Kerwin.

Future readers might also want to search for "file URI scheme RFC", and find the latest version. If you're a programmer, read the RFC. This post is to raise the awareness of the some of the issues around file to URI encoding, but it's not a substitution.

Recently I've been running into interop problems as some platforms are unable to parse file:/foo/bar. But this is not the first time I'm having trouble with file path represented as URI. Considering that the notion of filesystem goes back to 1960s, and URL has been around since 1990s, it's surprising that we haven't come to a concensus on this. But then again, like decimal numbers, once you start digging deeper, or start exchanging data, we find some glitches in the Matrix.

Gigahorse 0.3.0

in

Gigahorse 0.3.0 is now released. See documentation on what it is.

OkHttp support

0.3.0 adds Square OkHttp support. Gigahorse-OkHttp is availble for Scala 2.10, 2.11, and 2.12.

According to the JavaDoc you actually don't have to close the OkHttpClient instance.

scala> import gigahorse._, support.okhttp.Gigahorse
import gigahorse._
import support.okhttp.Gigahorse
 
scala> import scala.concurrent._, duration._
import scala.concurrent._
import duration._

herding cats: day 16

in

Contraband, an alternative to case class

in

Here are a few questions I've been thinking about:

  • How should I express data or API?
  • How should the data be represented in Java or Scala?
  • How do I convert the data into wire formats such as JSON?
  • How do I evolve the data without breaking binary compatibility?

limitation of case class

The sealed trait and case class is the idiomatic way to represent datatypes in Scala, but it's impossible to add fields in binary compatible way. Take for example a simple case class Greeting, and see how it would expand into a class and a companion object:

 

Gigahorse 0.2.0

in

Gigahorse 0.2.0 is now released. The new change is that it abstracts over two backends. @alexdupre contributed migration from AHC 1.9 to AHC 2.0, which is based on Netty 4 in #12.

In addition, there's now an experimental Akka HTTP support that I added. #15

Please see Gigahorse docs for the details.

gigahorse-github 0.1.0

in

gigahorse-github 0.1.0 is released. This is a Gigahorse plugin for Github API v3.

Here’s a quick example of how to get repository info:

scala> import gigahorse._, gigahorse.github.Github, scala.concurrent._, duration._
 
scala> val client = Github.localConfigClient
client: gigahorse.github.LocalConfigClient = LocalConfigClient(OAuthClient(****, List(StringMediaType(application/json), GithubMediaType(Some(v3),None,Some(json)))))
 
scala> Gigahorse.withHttp { http =>
         val f = http.run(client(Github.repo("eed3si9n", "gigahorse-github")), Github.asRepo)
         Await.result(f, 2.minutes)
       }
res0: gigahorse.github.response.Repo = Repo(https://api.github.com/repos/eed3si9n/gigahorse-github, gigahorse-github, 64614221, User(https://api.github.com/users/eed3si9n, eed3si9n, 184683, Some(https://github.com/eed3si9n), Some(https://avatars.githubusercontent.com/u/184683?v=3), Some(), Some(User), Some(true), None, None), eed3si9n/gigahorse-github, Some(Gigahorse plugin for Github API v3),...

If you're interested in gigahorse-github itself, README contains the full documentation.

extending Gigahorse

I also wrote Extending Gigahorse page describing the overview of how to write a Gigahorse plugin, which is more or less the same as how one would write a Dispach plugin. As I wrote there, the JSON data binding is auto generated from a schema.

For me, gigahorse-github was as much a proof of concept for sbt-datatype as it was for Gigahorse. It did end up exposing minor bugs on all components along the stack, so it was a fruitful exercise.

Gigahorse 0.1.0

in

Update: please use Gigahorse 0.1.1

Gigahorse 0.1.0 is now released. It is an HTTP client for Scala with Async Http Client underneath. Please see Gigahorse docs for the details. Here's an example snippet to get the feel of the library.

scala> import gigahorse._
scala> import scala.concurrent._, duration._
scala> Gigahorse.withHttp(Gigahorse.config) { http =>
         val r = Gigahorse.url("http://api.duckduckgo.com").get.
           addQueryString(
             "q" -> "1 + 1",
             "format" -> "json"
           )
         val f = http.run(r, Gigahorse.asString andThen {_.take(60)})
         Await.result(f, 120.seconds)
       }

registry and reference pattern

in

There's a "pattern" that I've been thinking about, which arises in some situation while persisting/serializing objects.

To motivate this, consider the following case class:

scala> case class User(name: String, parents: List[User])
defined class User
 
scala> val alice = User("Alice", Nil)
alice: User = User(Alice,List())
 
scala> val bob = User("Bob", alice :: Nil)
bob: User = User(Bob,List(User(Alice,List())))
 
scala> val charles = User("Charles", bob :: Nil)
charles: User = User(Charles,List(User(Bob,List(User(Alice,List())))))
 
scala> val users = List(alice, bob, charles)
users: List[User] = List(User(Alice,List()), User(Bob,List(User(Alice,List()))),
  User(Charles,List(User(Bob,List(User(Alice,List()))))))

The important part is that it contains parents field, which contains a list of other users.
Now let's say you want to turn users list of users into JSON.

[{ "name": "Alice", "parents": [] },
{ "name": "Bob",
  "parents": [{ "name": "Alice", "parents": [] }] },
{ "name": "Charles",
  "parents": [{ "name": "Bob", "parents": [{ "name": "Alice", "parents": [] }] }] }]

sjson-new and the prisoner of Azkaban

in

This is part 3 on the topic of sjson-new. See also part 1 and part 2.

Within the sbt code base there are a few places where the persisted data is in the order of hundreds of megabytes that I suspect it becomes a performance bottleneck, especially on machines without an SSD drive.
Naturally, my first instinct was to start reading up on the encoding of Google Protocol Buffers to implement my own custom binary format.

microbenchmark using sbt-jmh

What I should've done first, is start benchmarking. Using @ktosopl (Konrad Malawski)'s sbt-jmh, setting up a microbenchmark is easy. All you have to do is pop that plugin into your build. and create a subproject that enables JmhPlugin.

Syndicate content