Scala combinator library for working with binary data.
This library focuses on contract-first and pure functional encoding and decoding of binary data. The following design constraints are considered:
- Binary structure should mirror protocol definitions and be self-evident under casual reading
- Mapping binary structures to types should be statically verified
- Encoding and decoding should be purely functional
- Failures in encoding and decoding should provide descriptive errors
- Compiler plugin should not be used
As a result, the library is implemented as a combinator based DSL. Performance is considered but yields to the above design constraints.
The library uses Scalaz and Shapeless and is heavily influenced by scala.util.parsing.combinator.
The primary abstraction is a Codec[A]
, which supports encoding a value of type A
to a
BitVector
and decoding a BitVector
to a value of type A
.
The Codecs
object provides a number of predefined codecs and combinators.
import scodec._
import Codecs._
import scalaz.\/
// Create a codec for an 8-bit unsigned int followed by an 8-bit unsigned int followed by a 16-bit unsigned int
val firstCodec = (uint8 ~ uint8 ~ uint16)
// Decode a bit vector using that codec
val result: String \/ (Int ~ Int ~ Int) = Codec.decode(firstCodec, BitVector(0x10, 0x2a, 0x03, 0xff))
// Sum the result
val add3 = (_: Int) + (_: Int) + (_: Int)
val sum: String \/ Int = result map add3
Automatic case class binding is supported via Shapeless HLists:
import shapeless._
case class Point(x: Int, y: Int, z: Int)
implicit val pointIso = Iso.hlist(Point.apply _, Point.unapply _)
val pointCodec = (int8 :: int8 :: int8).as[Point]
val encoded: String \/ BitVector = pointCodec.encode(Point(-5, 10, 1))
// \/-(BitVector(24 bits, 0xfb0a01))
val decoded: String \/ Point = Codec.decode(pointCodec, BitVector(0xfb, 0x0a, 0x01))
// \/-(Point(-5,10,1))
Codecs can also be implicitly resolved, resulting in usage like:
// Assuming Codec[Point] is in implicit scope
val encoded = Codec.encode(Point(-5, 10, 1))
// \/-(BitVector(24 bits, 0xfb0a01))
val decoded = Codec.decode[Point](BitVector(0xfb, 0x0a, 0x01))
// \/-(Point(-5,10,1))
New codecs can be created by either implementing the Codec
trait or by passing an encoder function and decoder function to the Codec
apply method. Typically, new codecs are created by applying one or more combinators to existing codecs.
There are a number of built in combinators:
"name" | a
- creates aCodec[A]
that prefixes any error messages with the specified name.- Tuple Support
a ~ b
- creates aCodec[(A, B)]
that first decodesa
and thenb
.a ~> b
- creates aCodec[B]
that decodes first witha
and then withb
and throws away the decodeda
. For encoding, the zero value of typeA
s monoid is encoded.a <~ b
- creates aCodec[A]
that decodes first witha
and then withb
and throws away the decodedb
. For encoding, the zero value of typeB
s monoid is encoded.a >>~ f
- creates aCodec[(A, B)]
that decodes first witha
and then with the codec returned fromf(decodedA)
. The non-operator version of this isflatZip(f: A => Codec[B]): Codec[(A, B)]
.
- HList Support
a :: b
- creates aCodec[A :: B :: HNil]
or ifB
is an HList, aCodec[A :: B]
.a :~>: b
- creates aCodec[B]
(if B is an HList) that decodes and throws away the decodeda
and encodesa
s zero value.a >>:~ f
- creates aCodec[A :: B]
that decodes first witha
and then with the HList codec returned fromf(decodedA)
. The non-operator version of this isflatPrepend[L <: HList](f: A => Codec[L]): Codec[A :: L]
.l :+ a
- creates a codec that decodes first with the HList codecl
and then with the codeca
. This is the codec equivalent of HList append.k ::: l
- creates a codec that decodes first with the HList codeck
and then with the HList codecl
. This is the codec equivalent of HList concat.a.hlist
- creates aCodec[A :: HNil]
.
- Sizing
fixedSizeBits(size, a)
andfixedSizeBytes(size, a)
- creates aCodec[A]
that always encodes/decodessize
bits/bytes.variableSizeBits(sizeCodec, a)
andvariableSizeBytes(size, a)
- creates aCodec[A]
that encodes the size of the encodedA
followed by the encodedA
.
conditional(boolean, a)
- creates aCodec[Option[A]]
that skips encoding/decoding if the specified boolean is falserepeated(a)
- creates aCodec[IndexedSeq[A]]
- Type Conversions
a.xmap(f, g)
- creates aCodec[B]
given bidirectional functionsf: A => B
andg: B => A
.a.as[B]
- creates aCodec[B]
if ashapeless.Iso[B, A]
is in implicit scope
There are various examples in the test directory, including codecs for:
This library works with Scala 2.10.*.
The latest released version is 1.0.0-SNAP4, which is a stable snapshot of 1.0.0.
For SBT users:
libraryDependencies += "com.github.mpilquist" %% "scodec" % "1.0.0-SNAP4"
For Maven users:
<dependencies>
<dependency>
<groupId>com.github.mpilquist</groupId>
<artifactId>scodec_2.10</artifactId>
<version>1.0.0-SNAP4</version>
</dependency>
</dependencies>
Snapshot builds of the master branch are available on Sonatype's OSS hosting at https://oss.sonatype.org/content/repositories/snapshots/.
For SBT users:
resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots/"
libraryDependencies += "com.github.mpilquist" %% "scodec" % "1.0.0-SNAPSHOT"
For Maven users:
<repositories>
<repository>
<id>sonatype-oss-snapshots</id>
<name>Sonatype OSS Snapshots</name>
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.github.mpilquist</groupId>
<artifactId>scodec_2.10</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
</dependencies>
This project uses sbt. To build, run sbt publish-local
.