search term:

Coursera machine learning memo

This holiday break, I somehow got into binge watching Coursera’s Stanford Machine Learning course taught by Andrew Ng. I remember machine learning to be really math heavy, but I found this one more accessible.

Here are some notes for my own use. (I am removing all the fun examples, and making it dry, so if you’re interested in machine learning, you should check out the course or its official notes.)

encoding file path as URI reference

In this post I am going to discuss an old new problem of encoding file path as Uniform Resource Identifier (URI) reference.

As of 2017, the authoritative source of information is RFC 8089 - The “file” URI Scheme written by Matthew Kerwin.

Future readers might also want to search for “file URI scheme RFC”, and find the latest version. If you’re a programmer, read the RFC. This post is to raise the awareness of the some of the issues around file to URI encoding, but it’s not a substitution.

sbt server with Sublime Text 3

On Tech Hub blog I demonstrated how to use sbt server from VS Code to display compiler errors from a running sbt session. In this post, I’ll show how to do that for Sublime Text 3 in this post.

setting up Sublime Text 3 with sbt server

First, add tomv564/LSP plugin to Sublime Text 3.

  1. cd ~/Library/Application\ Support/Sublime\ Text\ 3/Packages
  2. git clone https://github.com/tomv564/LSP.git
  3. Run ‘Preferences > Package Control > Satisfy Dependencies’

Next, download sbt-server-stdio.js and save it to ~/bin/ or somewhere you keep scripts. sbt server by default uses Unix domain sockets on POSIX systems and named pipe on Windows, but editors seem to expect stdio. The script is a Node script that’s included as our VS Code extension that discovers the socket, and fronts it with stdio.

Ergodox

Over the weekend I assembled an Ergodox.

  • Infinity ErgoDox Ergonomic Keyboard Kit via massdrop
  • Cherry MX Brown switches
  • Datamancer Infinity Ergodox Hardwood Case (Black Walnut / Original) via massdrop
  • Plum Blossom PBT Dye-Subbed Keycap Set (OEM, Blank) via massdrop

Scala language server using sbt

It’s been a month since sbt 1.0 shipped, and I can finally sit back and think about sbt server again. Using my weekends time, I started hacking on an implementation of Scala language server on top of sbt server.

what is a language server?

A language server is a program that can provide language service to editors like Visual Studio Code, Eclipse Che, and Sublime Text 3 via Language Server Protocol. A typical operation might be textDocument/didOpen, which tells the server that a source file was opened in the editor.

Persistent Versioning

In this post, I’d like to introduce a version scheme that I call Persistent Versioning. Most of the ideas presented in this post are not new or my own. Let me know if there’s already a name for it.

In 2015, Jake Wharton (@JakeWharton) wrote a blog post titled Java Interoperability Policy for Major Version Updates:

auto publish (a website) from Travis-CI

GitHub Pages is a convenient place to host OSS project docs. This post explains how to use Travis CI to deploy your docs automatically on a pull request merge.

1. Generate a fresh RSA key in some directory

Make a directory outside of your project first. Pick a key name deploy_yourproject_rsa, so you can distinguish it from other keys.

$ mkdir keys
$ cd keys
$ ssh-keygen -t rsa -b 4096 -C "yours@example.com"
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/xxx/.ssh/id_rsa): deploy_website_rsa
Enter passphrase (empty for no passphrase):

    Read More…
  

tray for Atreus

In the last post that I wrote about Atreus build, I noted that there’s an issue of keyboard positioning:

Even if I can overcome the layout and memorize the various symbol locations, there’s the issue of the placement. If I place the keyboard in between me and the laptop the screen becomes too far.

I solved this issue by making a tray for Atreus that I can position it on top of the MacBook Pro keyboard.

Atreus

Last night I finished making my Atreus keyboard from a DYI kit that I got a while back. Here are some of the details:

  • I chose Matias Quiet Click switch option (gray slider). There’s no clicking.
  • The modifiers use Matias Quiet Linear switches (red slider).
  • There are 42 keys in split ortholinear layout.
  • Mahogany ply case.

The materials

The kit comes with almost everything you need to assemble the Arteus keyboard. You need lacquer, a soldering iron, solder, and wire cutters.

Gigahorse 0.3.0

Gigahorse 0.3.0 is now released. See documentation on what it is.

OkHttp support

0.3.0 adds Square OkHttp support. Gigahorse-OkHttp is availble for Scala 2.10, 2.11, and 2.12.

According to the JavaDoc you actually don’t have to close the OkHttpClient instance.

scala> import gigahorse._, support.okhttp.Gigahorse
import gigahorse._
import support.okhttp.Gigahorse

scala> import scala.concurrent._, duration._
import scala.concurrent._
import duration._

scala> val http = Gigahorse.http(Gigahorse.config) // don't have to close
http: gigahorse.HttpClient = gigahorse.support.okhttp.OkhClient@23b48158

downloading and running app on the side with sbt-sidedish

I’ve been asked by a few people on downloading JARs, and then running them from an sbt plugin. Most recently, Shane Delmore (@shanedelmore) asked me about this at nescala in Brooklyn.

During an unconference session I hacked together a demo, and I continued some more after I came home.

sbt-sidedish

sbt-sidedish is a toolkit for plugin authors to download and run an app on the side from a plugin. It on its own does not define any plugins.

Contraband, an alternative to case class

Here are a few questions I’ve been thinking about:

  • How should I express data or API?
  • How should the data be represented in Java or Scala?
  • How do I convert the data into wire formats such as JSON?
  • How do I evolve the data without breaking binary compatibility?

limitation of case class

The sealed trait and case class is the idiomatic way to represent datatypes in Scala, but it’s impossible to add fields in binary compatible way. Take for example a simple case class Greeting, and see how it would expand into a class and a companion object:

Gigahorse 0.2.0

Gigahorse 0.2.0 is now released. The new change is that it abstracts over two backends. @alexdupre contributed migration from AHC 1.9 to AHC 2.0, which is based on Netty 4 in #12.

In addition, there’s now an experimental Akka HTTP support that I added. #15

Please see Gigahorse docs for the details.

Gigahorse 0.1.0

Update: please use Gigahorse 0.1.1

Gigahorse 0.1.0 is now released. It is an HTTP client for Scala with Async Http Client underneath. Please see Gigahorse docs for the details. Here’s an example snippet to get the feel of the library.

scala> import gigahorse._
scala> import scala.concurrent._, duration._
scala> Gigahorse.withHttp(Gigahorse.config) { http =>
         val r = Gigahorse.url("http://api.duckduckgo.com").get.
           addQueryString(
             "q" -> "1 + 1",
             "format" -> "json"
           )
         val f = http.run(r, Gigahorse.asString andThen {_.take(60)})
         Await.result(f, 120.seconds)
       }

registry and reference pattern

There’s a “pattern” that I’ve been thinking about, which arises in some situation while persisting/serializing objects.

To motivate this, consider the following case class:

scala> case class User(name: String, parents: List[User])
defined class User

scala> val alice = User("Alice", Nil)
alice: User = User(Alice,List())

scala> val bob = User("Bob", alice :: Nil)
bob: User = User(Bob,List(User(Alice,List())))

scala> val charles = User("Charles", bob :: Nil)
charles: User = User(Charles,List(User(Bob,List(User(Alice,List())))))

scala> val users = List(alice, bob, charles)
users: List[User] = List(User(Alice,List()), User(Bob,List(User(Alice,List()))),
  User(Charles,List(User(Bob,List(User(Alice,List()))))))

    Read More…
  

sjson-new and the prisoner of Azkaban

This is part 3 on the topic of sjson-new. See also part 1 and part 2.

Within the sbt code base there are a few places where the persisted data is in the order of hundreds of megabytes that I suspect it becomes a performance bottleneck, especially on machines without an SSD drive. Naturally, my first instinct was to start reading up on the encoding of Google Protocol Buffers to implement my own custom binary format.

sjson-new and custom codecs using LList

Two months ago, I wrote about sjson-new. I was working on that again over the weekend, so here’s the update. In the earlier post, I’ve introduced the family tree of JSON libraries in Scala ecosystem, the notion of backend independent, typeclass based JSON codec library. I concluded that we need some easy way of defining a custom codec for it to be usable.

roll your own shapeless

In between the April post and the last weekend, there were flatMap(Oslo) 2016 and Scala Days New York 2016. Unfortunately I wasn’t able to attend flatMap, but I was able to catch Daniel Spiewak’s “Roll Your Own Shapeless” talk in New York. The full flatMap version is available on vimeo, so I recommend you check it out.

sjson-new

background

One of the fun way of thinking about software projects is literary analysis. Instead of the actual source code, think of who wrote it when and why (what problem does it solve), and how it’s written (what influenced it). Within the Scala ecosystem, not too many genre are as rich as the JSON libraries.

In December 2008, the first edition of Programming in Scala came out, which used JSON as an example in the context of parser combinator, and showed that JSON parser can be written in 10 lines of code:

sbt server reboot

This is a continuation from the sbt 1.0 roadmap that I wrote recently. In this post, I’m going to introduce a new implementation of sbt server. Please post on sbt-dev mailing list for feedback.

The motivation for sbt server is better IDE integration.

A build is a giant, mutable, shared, state, device. It’s called disk! The build works with disk. You cannot get away from disk.

– Josh Suereth in The road to sbt 1.0 is paved with server

The disk on your machine is fundamentally a stateful thing, and sbt can execute the tasks in parallel only because it has the full control of the effects. Any time you are running both sbt and an IDE, or you’re running multiple instances of sbt against the same build, sbt cannot guarantee the state of the build.

ScalaMatsuri as a lifestyle

For me (and for many of the 27 organizers, I imagine) ScalaMatsuri is a lifestyle. It’s true that there was a successful two-day conference in Tokyo with 550 participants. But for us the organizers, the preparation has been going on since February 28th, for 11 months. Despite the fact that my contribution was small, planning ScalaMatsuri 2016 was by far the most amount of involvement I’ve committed to. Through the course of planning months, there were many discussions over Slack, Hangouts, and occasionally even face-to-face. The fun part was coming up with the ideas together, and seeing them materialize. Sometimes, I was the one coming up with radical ideas that were being executed by someone else, while other times, it was the opposite case and I was getting my hands dirty.

stricter Scala with -Yno-lub

For a flexible language like Scala, it’s useful to think of subset of the programming language, like your own personal Good Parts, and opinionated style guides.

setup

To try -Yno-lub, you can drop in the following sbt plugin to project/ynolub.sbt:

addSbtPlugin("com.eed3si9n" % "sbt-ynolub" % "0.2.0")

lub

When Scala’s type inferencer finds type A and type B to unify, it tries to calculate the lub (least upper bounds) of two types with regards to <:<. This process is sometimes called lubbing. Here are some of the examples:

herding cats: day 1

Check out my new series herding cats. (I’m writing it using Pamflet from the get go)

This is a log of me going through Cats, a functional programming library for Scala that is currently experimental and under active development.

switching Java version

I’ve been switching between Mac and Ubuntu, and between Java 6 and 7 lately. This is a memo of how to switch Java versions on both Mac and Ubuntu.

Update: Yoshida-san told me about this thing called jEnv, which does all this.

Zshrc

Here’s one way of loading different shell files depending on the OS:

## basic
[ -f $HOME/dotfiles/zshrc.basic ] && source $HOME/dotfiles/zshrc.basic
 
## aliases
[ -f $HOME/dotfiles/zshrc.alias ] && source $HOME/dotfiles/zshrc.alias
 
case "${OSTYPE}" in
# MacOSX
darwin*)
  [ -f $HOME/dotfiles/zshrc.osx ] && source $HOME/dotfiles/zshrc.osx
  ;;
# Linux
linux*)
  [ -f $HOME/dotfiles/zshrc.linux ] && source $HOME/dotfiles/zshrc.linux
  ;;
esac
 
## color
[ -f $HOME/dotfiles/zshrc.color ] && source $HOME/dotfiles/zshrc.color

    Read More…
  

monads are fractals

On my way back from Uppsala, my mind wandered to a conversation I had with a collegue about the intuition of monads, which I pretty much butchered at the time. As I was mulling this over, it dawned on me.

Sierpinski triangle

monads are fractals

The above is a fractal called Sierpinski triangle, the only fractal I can remember to draw. Fractals are self-similar structure like the above triangle, in which the parts are similar to the whole (in this case exactly half the scale as parent triangle).

Monads are fractals. Given a monadic data structure, its values can be composed to form another value of the data structure. This is why it’s useful to programming, and this is why it occurrs in many situations.

Let’s look at some examples:

scala> List(List(1), List(2, 3), List(4))
res0: List[List[Int]] = List(List(1), List(2, 3), List(4))

towards universal access at a conference

Two days of #ScalaMatsuri ended as a huge success. But for the next year, I’m leaving ourselves a few homeworks to work on. As the title suggests, the next goal that we should aim for is universal access. In Scala language, universal access principle indicates the fact that both methods and fields can be accessed interchangeably from outside.

For a conference, I mean universal access to mean being more inclusive to various groups of people:

ScalaMatsuri day1

This year was the second Scala conference in Japan. We’ve changed the name to ScalaMatsuri, which means Scala festival in Japanese. 300 tickets sold out. With invited guests and free tickets for sponsors, there may have been even more people. The venue was at CyberAgent, which runs blog service and online ad services.

Day 1 kicked off with Martin’s (@ordersky) ‘Evolution of Scala.’ Many people were looking forward to see Martin, so I think it was full house from the get go. During all sessions that I attended, I was busy typing live text translation using my closed-captioning both from English to Japanese, and from Japanese to English along with other members @cbirchall, @cdepillabout, @okapies, and @oe_uia.

IRC memo

register your nick

/msg NickServ REGISTER %password% youremail@example.com

What is the recommended way to set up my IRC nickname?

make new channel

To check whether a channel has already been registered, use the command:

/msg ChanServ info ##channelname

/join ##channelname

The command to register your channel (once you’ve joined it and you have op status) is as follows:

/msg ChanServ register ##channelname

    Read More…
  
  

Vim memo

Personally, I don’t mind using SublimeText, which is my editor of choice for a while. But I’m also curious about commandline editors since many people taut their ability to code over the network. You could forward X or remote in using some other way and still use Sublime, but let’s see how if goes.

I started working on this Vim setup when I got a new MBP recently. Figured, I can try something new. So, this post is more of a personal memo written by a total newbie, which is what blogs are all about. caveat emptor. In general though, the configuration is mostly inspired by yuroyoro-san’s blog post from a couple years ago.

scripting with Scala

The need for regular expressions is real. Whenver I need to transform a set of text files it usually ends up with fumbling through the documentation of find command, zsh, and StackOverflow Perl questions. I would rather use Scala instead of muddling through Perl. It’s really the matter of my familiarity than anything else.

For example, I now have over a hundred reStructuredText files that I want to convert into markdown. I first tried pandoc, and it looked mostly ok. As I was going through the details, however, I noticed that many of the code literals were not converting over as formatted. This is because they were formatted using either single ticks or using Interpreted Text. Preprocessing the text with a series of regex replacements should work.

sbt technology preview: auto plugins

See Preview of upcoming sbt 1.0 features: Read about the new plugins posted on typesafe.com/blog.

nescala 2014 day 2: 30 sbt plugins in 15 minutes

Slides from nescala day 2 unconference:

I may have added a few more :)

learning Scalaz: nescala 2014

Here are the slide decks and video for learning Scalaz talk:

constraining class linearization (mixin order) in Scala

Woke up early yesterday, so I started skimming @xuwei_k’s override blog post. The topic was so intriguing, I got out of the bed and started translating it as the curious case of putting override modifier when overriding an abstract method in Scala. In there he describes the conundrum of providing the default instances to typeclasses by using Scalaz codebase as an example.

Here’s a simplified representation of the problem:

curious case of putting override modifier when overriding an abstract method in Scala

This is a translation of Scalaで抽象メソッドをoverrideする際にoverride修飾子を付けるべきかどうかの是非 by Kenji Yoshida (@xuwei_k), a Scalaz committer.

First, a quote from Programming in Scala, 2nd ed. p. 192:

Scala requires [override] modifier for all members that override a concrete member in a parent class. The modifier is optional if a member implements an abstract member with the same name.

In this post, we’ll discuss this “The modififier is optional.” Since overriding an existing method with implementation requires override modifier, and failure to do so would result to a compiler error, there’s not much to talk about for that case. We’ll focus on whether one should put override modifier or not in the case of overring an abtract method. I don’t think there’s going to be any difference in Scala version, but let’s assume the latest stable 2.10.3.

sequencing tasks with sbt-sequential

In this post, I will discuss the execution semantics and task sequencing in sbt 0.13. First we will cover the background, and then I will introduce a new experimental plugin sbt-sequential that adds sequential tasks.

background

Mark said:

The sbt model is to have your side effects be local to your task so that as long as dependencies are satisfied, the task can be executed whenever. The win is parallel by default and enabling faster builds in practice.

traveling through the 4th dimension with sbt 0.13

Warning: This is a memo about sbt for intermediate users.

setting system

At the heart of sbt 0.13 is the setting system, just like sbt 0.12. Let’s look at Settings.scala:

trait Init[Scope] {
  ...

  final case class ScopedKey[T](
    scope: Scope,
    key: AttributeKey[T]) extends KeyedInitialize[T] {
    ...
  }

  sealed trait Initialize[T] {
    def dependencies: Seq[ScopedKey[_]]
    def evaluate(map: Settings[Scope]): T
    ...
  }

  sealed class Setting[T] private[Init](
    val key: ScopedKey[T], 
    val init: Initialize[T], 
    val pos: SourcePosition) extends SettingsDefinition {
    ...
  }
}

    Read More…
  

what is object-oriented programming?

How do you define oop?

purely object-oriented programming

Purely object-oriented programming is defined to be:

programming with objects.

What’s an object?

It is an atom that can hold references to other objects, receive predefined list of messages, and send messages to other objects and itself; and nothing else. A message consists of a name and a list of reference to objects.

This is it. The wording is mine, but the idea is from Alan Kay (2003), the guy who coined the term object-oriented programming. Anything else is either not directly tied to oop or an implementation detail.

sbt-logo proposal

See sbt-logo proposals thread on sbt-dev and the original thread on sbt. The source is at eed3si9n/sbt-logo.

Edit: Changed the tail of ’t’. See github for the original.

.sbt build definition

sbt is a build tool for Scala and Java projects that aims to do the basics well.

sbt is a build tool for Scala and Java projects that aims to do the basics well.


> ~compile


.sbt build definition

sbt is a build tool for Scala and Java projects that aims to do the basics well.

sbt is a build tool for Scala and Java projects that aims to do the basics well.

> ~compile


.sbt build definition

sbt is a build tool for Scala and Java projects that aims to do the basics well.

sbt is a build tool for Scala and Java projects that aims to do the basics well.

> ~compile

Scala: the flying sandwich parts

JavaScript existed since 1995 long before ‘JavaScript: The Good Parts’ (2008), jQuery (2006), and V8 (2008) happened. The interesting thing about Douglas Crockford’s ‘The Good Parts’ is that unlike the other additive work, it’s a book about subtracting features from the language.

I’ve been thinking about exploring a subset of Scala in a wonderland setting without the “real world” constraints such as Java familiarity and interoperability. If using Scala as an alternative Java is acceptable, why not try using it as an alternative functional programming language? Another point of this thought experiment is to see some of the duplicate constructs can be reduced. In this article, I’m not interested in finding out the idiomatic way, or calling something good or bad. I’m calling this The Flying Sandwich Parts (TFSP).

scopt 3.0

scopt is a little command line options parsing library.

Today, I’m releasing scopt 3.0. If you’re not interested in the implementation details, skip to the readme.

Around March 4th, 2010, I became a committer to scopt, a fork of Aaron Harnly’s scala-options that was written in 2008. I think I wanted to make a few changes around the usage text, key=value options, and argument list. Since then I’ve been fielding all the bug reports, including the request to publish the jar on scala-tools.org. On March 18, 2012, I forked the project again to scopt/scopt and released scopt 2.0.0 that added immutable parser.

how to write a Dispatch plugin

Dispatch has been the de facto library to get to the Internet from Scala. To keep in step with the recent move towards non-blocking IO, @n8han rewrote the library as Reboot based on Async Http Client. This became Dispatch 0.9. Then Dispatch 0.10 came out to replace its own Promise type with the standarized SIP-14 Future.

As with Dispatch Classic, Reboot lets you write plugins to wrap around web APIs. In this post we’ll port a plugin from Classic to explore how it works.

working on your own twitter bot?

Notes on 'Monads Are Not Metaphors'

This is a translation of 「モナドはメタファーではない」に関する補足 by Kenji Yoshida (@xuwei_k), one of the most active Scala bloggers in Japan covering wide range of topics from Play to Scalaz.

Daniel Spiewak’s Monads Are Not Metaphors was written about two and a half years ago, but seeing how its Japanese translation is still being tweeted and being bookmarked by 250 users on Hantena, its popularity doesn’t seem to cease. I just remembered something to note about the example code used in the post, which could be an unstylish critique, but I’m going to jot it down here. It’s an unstylish critique, because I’ll be digging into the part where the author likely knew from the beginning but omitted it intentionally for the sake of illustration. Also I’ll be using Scalaz in this post.