These are notes of the talk I gave at ClojuTre 2019. There is a video recording of the talk, and the slide deck as a PDF.

This work is licensed under a “CC BY SA 4.0” license.

Hello ClojuTre. I'm Oleg from Helsinki.

Imagine you are writing a cool new rogue-like game. So cool many have no idea what's going on. A definitive character of rogue-likes is procedural generation of content. You'll need a random number generator for that.

SplitMix is a fast, splittable pseudorandom number generator. Being able to split a generator into two independent generators is a property you'll want if you use the generator in a functional programming language. Look it up. We'll concentrate on the being fast part. Obviously you want things to be fast.

SplitMix is fast. It does only 9 operations per generated number, one addition to advance the random seed, and `xor`

, `shift`

, `multiply`

`xor`

, `shift`

, `multiply`

, `xor`

and `shift`

. 9 in total.

However, for the maximum reach, we want our game to run in the web browsers.

- JavaScript is a great platform
- But a terrible programming language

Look carefully. When we multiply two "big" odd numbers, we do get even result. We shouldn't: product of two odd numbers is odd.

JavaScript numbers are a mess. Next a bit of arithmetics.

Instead of multiplying two big numbers, we split them in high and low parts and multiply many small numbers. Like in an elementary school. Then we'll have enough precision.

So as we are multiplying 32bit numbers, and are interested only in lower 32 bits of 64 bit result. Then we need to do only three multiplications.

At the end, we can get correct results, i.e. good random numbers even in JavaScript.

Next if we replace the multiplication in `xor-shift-multiply`

with a macro doing the right thing, and actually change everything to be a macro, and expand we'll get...

a lot of code which barely fits on the slide.

I had to shorted `bit-shift-left`

to `bsl`

and `unsigned-bit-shift-right`

to `ubsr`

.

Look closely. Would you write this kind of code.

We bind ("assign") a constant value to `uv`

variable.

And the calculate the high and low 16 bit parts of it. Something we could write directly: `let [u 0x85eb v ca6b]`

, i.e. *optimize by hand*. Should we do so?

No. Let's rather write an (optimizing) compiler. We don't have time to optimize by hand.

Recall, we have a very specific problem. Working with a very very tiny subset of Clojure. Some simple arithmetic and bit mangling of 32bit numbers.

We'd like to add a little of `magic`

, which would make the program magically run faster. In my toy-micro benchmarks the speed-up is 10 percent. It's definitely worth it.

What is this magic?

A little cute macro. Of course.

We get a `form`

and convert it into internal representation on a way in, and back to Clojure on the way back.

Working the whole Clojure syntax directly is insane task. We want only deal with a small sane subset of it. Also in a easier to manipulate format. Clojure as a pragmatic language is still optimized for writing and reading, not machine manipulation so much.

The next step is to expand a multiplication using a trick we have seen previously. We could have done it already in `from-clojure`

step, but it's good engineering to have only one thing per step.

And the important part is the `optimise`

function.

The internal representation I used is nested vectors with a node type as a first element. An uniform representation made it way easier to do everything else. Note how literals 1 and 3 are wrapped into vector too. We can simply look at the head of a vector to know what we are dealing with.

Code is Data.

A difficult part in *Code is Data* are local variables. I chose to use *de Bruijn* indices, so instead of names: `x`

and `y`

or `a`

and `b`

there are numbers counting towards the corresponding `let`

(which binds only one variable at the time by the way).

de Bruijn indices are tricky to grok. Look at the colors, they are there to help. The blue `:var 1`

references "one away" `let`

.

I don't expect you to understand them. It's one way to represent bindings. Perfectly there should be a library, so you don't need to think about low-level details.

Once we got rid out of names, we can still keep them around using metadata. That's a really cool feature in Clojure, I have to admit. The metadata is there, but not in your way. And having names around is actually useful when you try to debug things.

Now we have a setup done. Let's jump into optimizations.

Recall our code snippet. There's `uv`

which is bound to a constant. And then it's used an expression which could simplify if we do this and that...

Let's keep it super simple.

- We can have a small set of simple local rewrite rules. Local meaning, we don't need to look around, only at one subexpression at the time.
- Then we try to match the rule everywhere, and if it match, perform the rewrite.
- And loop until there's nothing to do.

The first optimization is inlining. In a sense it's most powerful one, as it makes opportunities for other optimizations to fire, even that on itself it doesn't do much.

So if we have a let-binding, then in some cases we perform a substitution. Replace all `x`

s with an `expression`

inside a body.

`let x 1 y 1 (+ x y)`

to a lot simpler `(+ 1 2)`

. But nothing more, just that.

I need to point out, that optimizing is somewhat of an art. Sometimes it work, sometimes it don't.

For example, we don't want to duplicate an expensive `(fibonacci 100)`

expression. We want to evaluate it once and share the result.

On the other hand, if someone already went and computed the value, then we can push it to the leaves of an expression tree.

Heuristics are tricky.

Luckily for our needs simple heuristics work well.

When *inlining* is a valid rewrite?

Not in every language. Consider this not-so-functional example.

If we substitute everything, the `(do-foo)`

and `(do-bar)`

would be in different order. And `(do-quux)`

will be gone completely, hopefully it didn't anything important!

We need a language where there are no side-effects, nor there are so much difference in the execution order. Whole Clojure isn't such language. Our tiny subset is.

I have heard there are programming languages which behave like our small one, but are more general purpose!

OK. Let's move to the next optimization.

When we have something simple as `(+ 1 2)`

, let us evaluate it already at compile time.

For every primitive operation, if the arguments are known constants, just do it.

Now, I ask you when *constant folding* is a valid rewrite.

Well, that was a trick question. It really depends on primitives, whether it make sense to perform them at compile time. (Even pure languages have primitives to print stuff on a screen).

But you could think about that `precalculate`

example. When does it makes to perform the calculation (assuming we somehow know it terminates):

- At compile time?
- At start up?
- At first access?

It really depends, and there are no single simple answer.

Again, optimizations is an art.

Because we work with nice internal representation, writing individual optimizations is so nice.

- if a node is not one of special nodes
- and all node arguments are constants
- evaluate it.

The code is shorter than my explanation. And still understandable.

With these two optimizations, inlining and constant folding, we get from this big (and repetitive) code blob to...

Something which actually fits on the slide without font size scaling. It's not super-pretty, but it's hard to spot if something can be done there.

We can make it look nice with one more optimization. When you bind a let-expression to a variable in outer let-expression, we can float out the inner one.

A very old idea it is.

And then we get (in my opinion) a very nice direct code. Six intermediate results to get final one.

That's what I wanted to tell you.

- Implementing small (domain specific) languages is fun.
- If you approach problems with "let's write a programming language to describe them" -attitude there a lot of big hammers in your disposal. A lot of wheels is already invented.
- Languages don't need only be about numerics, it could be HTTP routing, authorisation rules, UI-workflows, CI-scripts (I could bash about bash), data descriptions, you name it.
- But make your languages
*typed, lazy, pure, total or and even dependent*for extra fun and interesting new problems. ;)

Thank you.

]]>As `Cabal-3.0.0.0`

is now released, I uploaded the `cabal-fmt`

tool to Hackage. I have been using `cabal-fmt`

for over a half year now for my own Haskell projects, and have been happy with this minimal, yet useful tool. `cabal-fmt`

formats `.cabal`

file preserving the field ordering and comments.

`cabal-fmt`

is based on `Distribution.Fields`

functionality. `cabal-fmt`

is a thin addition on top. Same `Distribution.Fields`

(and related `Distribution.FieldsGrammar`

) is also used in `haskell-ci`

to parse and print `.cabal`

-like files. I also use it in other tools to implement configuration files. In my opinion the lexical structure of `.cabal`

files is more flexible and human-writing-friendly than YAML or JSON. YMMV. For example the header for this post is written as

with quotes needed to disambiguate YAML. That's silly :) Cabal-like syntax would be

However, enough bashing YAML.

`cabal-fmt`

is opinionated tool, it does format few fields to my liking. Let us see how.

`build-depends`

modules are formatted comma first (with `cabal-version: 2.2`

also with a leading comma), tabulated, sorted, and `^>=`

preferred when it can be used. For example:

```
build-depends:
, base ^>=4.11.1.0 || ^>=4.12.0.0
, bytestring ^>=0.10.8.2
, Cabal ^>=3.0.0.0
, containers ^>=0.5.11.0 || ^>=0.6.0.1
, filepath ^>=1.4.2
, mtl ^>=2.2.2
, parsec ^>=3.1.13.0
, pretty ^>=1.1.3.6
```

or (for older `cabal-version`

):

Single `build-depends`

are formatted as a single line, like

`exposed-modules`

, `other-modules`

, `default-extensions`

and `other-extensions`

are sorted and duplicates removed. For example.

```
exposed-modules:
CabalFmt
CabalFmt.Comments
CabalFmt.Error
CabalFmt.Fields
CabalFmt.Fields.BuildDepends
CabalFmt.Fields.Extensions
CabalFmt.Fields.Modules
CabalFmt.Fields.TestedWith
CabalFmt.Monad
CabalFmt.Options
CabalFmt.Parser
CabalFmt.Refactoring
```

Sometimes, you'll prefer some module to be the first, for `cabal repl`

. In that case I would use two `exposed-modules`

fields.

`tested-with`

is one more field where I don't like the default formatting either. This field drives the job selection in `haskell-ci`

. `cabal-fmt`

combines version ranges for compilers, and prints `GHC`

and `GHCJS`

in upper case.

The line generated is long, especially for packages supporting a lot of GHC versions. Something I don't have a clear preference yet how to handle.

The recent addition is an ability to (re)write field contents, while formatting. There's an old, ongoing discussion of allowing wildcard specification of `exposed-modules`

in `.cabal`

format. I'm against that change. Instead, rather `cabal-fmt`

(or an imaginary IDE), would regenerate parts of `.cabal`

file given some commands.

`cabal-fmt: expand <directory>`

is a one (the only at the moment) such command.

`cabal-fmt`

will look into *directory* for files, turn filenames into module names and append to the contents of `exposed-modules`

. As the field is then nubbed and sorted, expanding is idempotent. For example `cabal-fmt`

itself has:

The functionality is simple. There is no removal of `other-modules`

or `main-is`

. I think that using different directory for these is good enough workaround, and may make things clearer: directory for public modules and a directory for private ones.

And that's all that `cabal-fmt`

does. Formatting of other fields comes directly from `Cabal`

. I have few ideas, what else can be done, e.g.

`cabal-fmt: expand`

for`extra-source-files`

- formatting of
`reexported-modules`

- sorting of fields, e.g. putting
`type`

and`default-language`

to the top of the component stanzas.

But these don't bother me enough yet, so they are not there.

The implicit goal of a project is to iterate independently of `cabal-install`

, Find out what could be useful, and how it can be done, and later merge into `cabal-install`

's `cabal format`

functionality. Yet then providing enough configuration knobs to not be so opinionated.

Discussion around (limitations of) `Coercible`

, `GeneralizedNewtypeDeriving`

or `DerivingVia`

resume once in a while.

One common question is whether `fmap coerce = coerce`

should be a law of the `Functor`

type-class, or not. Intuitively, it's similar to the identity law: `fmap id = id`

law, but intuition is not good enough reason alone.

The identity law comes from category theory, so what could category theory say about `fmap coerce = coerce`

like proposition?

Let us consider an endofunctor from a three element category to itself, which "rotates the category".

maps objects to respectively. Careful reader would ask how maps morphisms. See an appendix, for a justification.

To make it more concrete we can think that as:

with isomorphisms in between (assuming `Int`

and `Word`

have the same amount of bits).

Then, we know that `Coercible B B'`

. And arrows and on the diagram are `coerce @B @B'`

and `coerce @B' @B`

. But `Coercible (F B) (F B') = Coercible B' A`

doesn't exist. is a function like `\(B' w) -> fromIntegral w`

which is not `coerce`

.

The functor doesn't preserve the `Coercible`

relation, even it's a categorically lawful functor.

But the functor above is not a `Functor`

as in Haskell. Not even if we try to define

we'll not be able to write `fmap :: (a -> b) -> F a -> F b`

, as `b`

can be anything, not only `A`

, `B`

or `B'`

.

`fmap`

is *polymorphically parametric*. There are papers and blogs explaining parametricity^{1}.

The key insight of parametricity is that we can read *types as relations*. For example a list defined as

can be read as a relation constructor:

Given a relation `R :: Relation X Y`

, `xs :: List X`

and `ys :: List Y`

are related with `List R :: Relation (List X) (List Y)`

, if either

`xs = Nil`

and`ys = Nil`

, or`xs = Cons x xs'`

,`ys = Cons y ys'`

, and`x`

and`y`

are related by`R`

, and`xs'`

and`ys'`

are recursively related by`List R`

.

In other words, `List R`

relates lists of equal length, when all elements are related pointwise.

We can derive free theorems for polymorphic functions over lists, or ...

We could take `R X Y`

to be `Coercible X Y`

. Relation doesn't need to be functional. `Coercible`

is a binary-relation. We can reason that `List Coercible`

is `Coercible`

for lists: Lists related by `coerce @(List X) @(List Y)`

are of the same length and each element is related by `coerce @X @Y`

.

Maybe instead of asking whether `fmap coerce = coerce`

should be a law of the `Functor`

type-class, we should ask whether `Functor`

should be a type-class of *parametric* type constructors? Parametricity would imply `fmap coerce = coerce`

. Non-requirement of parametricity, would mean that opaque types can be "lawfully" `Functor`

, e.g. similarly as `Set a`

has an `Eq`

instance.

The latter choice allows us to write type internally used in the `bifunctors`

library:

```
data Mag a b t where
Pure :: t -> Mag a b t
Map :: (x -> t) -> Mag a b x -> Mag a b t
Ap :: Mag a b (t -> u) -> Mag a b t -> Mag a b u
One :: a -> Mag a b b
instance Functor (Mag a b) where
fmap = Map
instance Applicative (Mag a b) where
pure = Pure
(<*>) = Ap
```

`Mag`

is not lawful `Functor`

, it violates `fmap id = id`

property. but is used to implement `traverseBiaWith`

.

However, there is a comment in the code:

```
-- Note: if it's necessary for some reason, we *could* relax GADTs to
-- `ExistentialQuantification` by changing the type of One to
--
-- One :: (b -> c) -> a -> Mag a b c
```

Then `Mag`

will be `Coercible`

in all argument, current `Mag`

has `type role Mag representational nominal nominal`

.

A third alternative is to use

GHC is smart enough to infer that this `Mag`

variant is also `representational`

in all arguments. Unfortunately, to implement `traverseBiaWith`

, we'll need to change the required constraints

```
- Biapplicative p
+( Biapplicative p
+, forall u v x y. (Coercible u x, Coercible v y)
+ => Coercible (p u v) (p x y))
+)
```

... or require `Bifunctor`

(a superclass of `Biapplicative`

) to be parametric in its both arguments!

For me, it looks like that the addition of `(forall x y. Coercible x y => Coercible (f x) (f y))`

constraint to `Functor`

(and similarly for `Bifunctor`

, `Profunctor`

, `Contravariant`

...) can be worked around in *all cases*. The `Mag`

is the only example mentioned as *useful* but not lawful `Functor`

, ^{2} and we have demonstrated a required change.

Note: `Mag`

will still be unlawful, but it can be made `representational`

in all arguments. In the implementation of `traverseBiaWith`

we will use `coerce`

which is free operationally, so there shouldn't be any performance degradation.

I cannot come up with an example of `f :: Type -> Type`

which would violate `fmap id = id`

law, but obey `fmap coerce = coerce`

one (or an opposite direction). And I cannot show that `fmap coerce = coerce`

follows from other two `Functor`

laws, but without using parametricity. Edward Kmett explains how `Functor`

composition law follows from identity law and parametricity. Coerce law is implied by parametricity directly.

Finally, my conclusion is that both

`fmap coerce = coerce`

should be a law of GHC Haskell`Functor`

, and- the
`forall a b. Coercible a b => Coercible (f a) (f b)`

be`Functor`

super-constraint

even the change departs us far far from Haskell2010. The justification is that these requirements are implied by *parametricity*, and `Functor`

class should contain only parametric type constructors. We would still be able to have unlawful `Functor`

s if really needed.

Some immediate benefits are ability to put `join`

into `Monad`

or that van Laarhoven `Lens s t a b`

would be `representable`

in all arguments.

The disadvantage is that there's Haskell 2010 code which would be broken by that change. Consider

We can write (unlawful) `Functor`

instance in Haskell 2010, which won't be accepted, as `Foo`

role is `nominal`

. Yet, GHC documentation about `DatatypeContexts`

says *This is widely considered a misfeature, and is going to be removed from the language.* You need to **enable** a language feature to write `Foo`

, in other words that code is already broken (since November 2010!)

Also a non-Haskell2010 code will be broken:

You may think, there's an easy solution, but there is a five year old issue about roles for type families. Also it have to be investigated, if someone actually wrote an (unlawful) `Functor`

wrapping a type family which pattern matches (i.e. is non-parametric) on an argument, and why they did that!

Just adding `fmap coerce = coerce`

law wouldn't break any code. Something which is unlawful will be a bit more unlawful. The hypothesis is that `fmap coerce = coerce`

won't make any currently lawful `Functor`

s into unlawful one.

I'm thankful to Ryan Scott for many discussions and valuable insights. And to Andres Löh for comments on a draft of this post.

In this category there is an unique arrow between any two objects, so the mapping is trivial.

We can extend the category by adding more objects and morphisms, and so it keeps all other objects in place. In this case mapping of morphisms is trickier. If we require that and others identities suggested by names or arrows we can make it work.

Let us define two families of morphisms indexed by objects in : and

Using these, we can define mapping of morphisms as

For example is mapped to

Check by looking at the diagram!

The identity and composition properties hold, for example

Some links about parametricity

↩For example in https://ryanglscott.github.io/2018/06/22/quantifiedconstraints-and-the-trouble-with-traversable/↩

This post contains excerpts from a PDF "post" I have written. It looks way better in PDF.

*Partial ordered set*, or poset for short is well used example in category theory books. Yet, posets are not too familiar for an average CT-curious Haskeller, yet they are easy to visualise! Let’s do exactly that, and mention elementery bits of category theory. In particular, I’ll scan over the six first chapters of *Category Theory* by Steve Awodey, repeating related definitions and poset examples.^{1}

**Definition** (Awodey 1.1) A *category* consist of the following data

Objects:

Arrows:

For each arrow , there are given objects

called the

*domain*and*codomain*of . We writeto indicate that and .

Given arrows and , that is, with

there is given an arrow

called the

*composite*of and .For each object , there is given an arrow

called the

*identity arrow*of .Associativity:

for all , , .

Unit:

for all .

We’ll see how `Category`

type-class is related later, in later section.

…

Before proceeding, we’ll answer a question: is there a category with posets as objects? Yes, it’s called ! What are the arrows in this category? An arrow from a poset to a poset is a function

that is *monotone*, in the sense that, for all ,

We need to know that is monotone, and also that if and are monotone, then is monotone. That holds, check!

Recall, posets are categories, so monotone functions are "mappings" between categories. A "homomorphism^{2} of categories" is called a functor.

**Definition** (Awodey 1.2) A *functor*

between categories and is a mapping of objects to objects and arrows to arrows, in such a way that

,

,

.

That is, preserves domains and codomains, identity arrows, and composition. A functor thus gives a sort of "picture" – perhaps distorted – of in .

The version looks quite different:

There is a mismatch of a notation of category theory, and what can be written as code. In CT notation acts both on objects and arrows. In `f`

acts on objects, and `fmap f`

acts on arrows. Substituting above, `id`

and `.`

into definition of functor, will also give familiar laws

However, `Functor`

-class is only for functors from a pseudo-category to itself, where `f`

, a mapping from types to types is a type-constructor, not arbitrary type family. `Functor`

is a very special case of category theoretical variant.

With small posets, like `Bool`

and `M2`

we can visualise a monotone function, a functor. Let’s consider a function

The graph of `f`

is on [fig:f-bool-to-m2]. Dotted and dashed lines are arrows in `Bool`

and `M2`

respectively. We can see on figure, that `f`

indeed gives a picture of `Bool`

in `M2`

.

In we have only written a mapping of objects, `True`

and `False`

. The mapping of arrows is something we need to check, to be able to claim that `f`

is a functor, and therefore a monotone function. The other way around, there are mappings from `Bool`

to `M2`

which aren’t monotone, and aren’t functors.

In this section we went backwards. More principal approach would been to first consider functors between poset categories. The monotonicity requirement is implied by first functor requirement. This is a power of category theory. When you look for something, category theory tells you which properties it should have. Once you find something which satisfies the requirements, you know that it’s the right one (up to an isomorphism).

…

Exponential lattices with `M2`

are pretty. `ZHO -> M2`

([fig:zho2m2]) has nice planar graph. . Yet the final stress test is `(ZHO -> ZHO) -> ZHO`

, or is on [fig:big]. A beautiful monster.

In Multi-stage docker build of Haskell webapp blog post I briefly mentioned `data-files`

. They are problematic. A simpler way is to use e.g. `file-embed-lzma`

or similar functionality to *embed data* into the final binary.

You can also embed *secret data* if you first *encrypt* it. This would reduce the pain when dealing with (large) secrets. I personally favor configuration (of running Docker containers) through environment variables. Injecting extra data into containers is inelegant: that's another way to "configure" running container, when one would be enough.

In this blog post, I'll show that dealing with encrypted data in Haskell is not too complicated. The code is in the same repository as the previous post. This post is based on Tutorial: AES Encryption and Decryption with OpenSSL, but is updated and adapted for Haskell.

To encrypt a plaintext using AES with OpenSSL, the `enc`

command is used. The following command will prompt you for a password, encrypt a file called `plaintext.txt`

and Base64 encode the output. The output will be written to `encrypted.txt`

.

This will result in a different output each time it is run. This is because a different (random) salt is used. The *Salt* is written as part of the output, and we will read it back in the next section. I used `HaskellCurry`

as a password, and placed an encrypted file in the repository.

Note that we use `-pbkdf2`

flag. It's available since OpenSSL 1.1.1, which *is* available in Ubuntu 18.04 at the time of writing. Update your systems! We use 100000 iterations.

The choice of SHA1 digest is done because `pkcs5_pbkdf2_hmac_sha1`

exists directly in `HsOpenSSL`

. We will use it to derive key and IV from a password in Haskell. Alternatively, you could use `-p`

flag, so `openssl`

prints the used Key and IV and provide these to the running service.

To decrypt file on command line, we'll use `-d`

option:

This command is useful to check "what's there". Next, the Haskell version.

To decrypt the output of an AES encryption (aes-256-cbc) we will use the `HsOpenSSL`

library. Unlike the command line, each step must be explicitly performed. Luckily, it's a lot nice that using C. There 6 steps:

- Embed a file
- Decode Base64
- Extract the salt
- Get a password
- Compute the key and initialization vector
- Decrypt the ciphertext

To embed file we use *Template Haskell*, `embedByteString`

from `file-embed-lzma`

library.

```
{-# LANGUAGE TemplateHaskell #-}
import Data.ByteString (ByteString)
import FileEmbedLzma (embedByteString)
encrypted :: ByteString
encrypted = $(embedByteString "encrypted.txt")
```

Decoding Base64 is an one-liner in Haskell. We use `decodeLenient`

because we are quite sure input is valid.

```
import Data.ByteString.Base64 (decodeLenient)
encrypted' :: ByteString
encrypted' = decodeLenient encrypted
```

Note: `HsOpenSSL`

can also handle Base64, but doesn't seem to provide lenient variant. `HsOpenSSL`

throws exceptions on errors.

Once we have decoded the cipher, we can read the salt. The Salt is identified by the 8 byte header (`Salted__`

), followed by the 8 byte salt. We start by ensuring the header exists, and then we extract the following 8 bytes:

```
extract
:: ByteString -- ^ password
-> ByteString -- ^ encrypted data
-> IO ByteString -- ^ decrypted data
extract password bs0 = do
when (BS.length bs0 < 16) $ fail "Too small input"
let (magic, bs1) = BS.splitAt 8 bs0
(salt, enc) = BS.splitAt 8 bs1
when (magic /= "Salted__") $ fail "No Salted__ header"
...
```

Probably you have some setup to extract configuration from environment variables. The following is a very simple way, which is enough for us.

We use `unix`

package, and `System.Posix.Env.ByteString.getEnv`

to get environment variable as `ByteString`

directly. The program will run in Docker in Linux: depending on `unix`

is not a problem.

```
{-# LANGUAGE OverloadedStrings #-}
import System.Posix.Env.ByteString (getEnv)
import OpenSSL (withOpenSSL)
main :: IO ()
main = withOpenSSL $ do
password <- getEnv "PASSWORD" >>= maybe (fail "PASSWORD not set") return
...
```

We also initialize the OpenSSL library using `withOpenSSL`

.

Once we have extracted the salt, we can use the salt and password to generate the Key and Initialization Vector (IV). To determine the Key and IV from the password we use the `pkcs5_pbkdf2_hmac_sha1`

function. PBKDF2 (Password-Based Key Derivation Function 2) is a key derivation function. We (as `openssl`

) derive both key and IV simultaneously:

```
import OpenSSL.EVP.Digest (pkcs5_pbkdf2_hmac_sha1)
iters :: Int
iters = 100000
...
let (key, iv) = BS.splitAt 32
$ pkcs5_pbkdf2_hmac_sha1 password salt iters 48
...
```

With the Key and IV computed, and the ciphertext decoded from Base64, we are now ready to decrypt the message.

```
import OpenSSL.EVP.Cipher (getCipherByName, CryptoMode(Decrypt), cipherBS)
...
cipher <- getCipherByName "aes-256-cbc"
>>= maybe (fail "no cipher") return
plain <- cipherBS cipher key iv Decrypt enc
...
```

In this post we embedded an encrypted file into Haskell application, which is then decrypted at run time. The complete copy of the code is at same repository, and changes done for this post are visible in a pull request.

]]>I wonder: if we have a decidable partial ordering: like `leq`

method in `PartialOrd`

type class in `lattices`

package.

Can we *sort* a list using `leq`

so we get kind of a topological sorting?

Note, topological sorting is used for graphs: there we don’t have `leq`

-like luxury. We’d first need to compute a transitive closure of a graph. On the other hand, given a list of "node", it may not include all nodes; and may include duplicates.

Insertion sort is a simple sorting algorithm, trivial to implement for lists (i.e. not in-place). I continue wondering: maybe it will produce topological ordering. Even insertion sort is simple, I still had an uneasy feeling: *does it really work*. Insertion sort terminates despite what function you pass as a comparator (even impure random one!), so what kind of output it produces when given *lawful* `leq`

?

I must admit that exposure to Haskell and Agda made me suspect all pen and paper proofs, especially made by myself. So let’s certify insertion sort. The plan is to show how to certify insertion order first for *total order* and then for *partial order*.

Module definitions and imports:

```
module Topo where
open import Data.List
open import Relation.Nullary
open import Relation.Binary.PropositionalEquality
```

Next, the definition of `insert`

and `sort`

. These are straight-forward and are familiar. Note that `_≤?_`

*decides* whether `≤`

-relation holds. Returning `Dec`

doesn’t forget what we compute (it returns the evidence), compared with simply returning `Bool`

.

```
module Sort {A : Set} (_≤_ : A → A → Set)
(_≤?_ : (x y : A) → Dec (x ≤ y))
where
insert : A → List A → List A
insert x [] = x ∷ []
insert x (y ∷ ys) with x ≤? y
... | yes x≤y = x ∷ y ∷ ys
... | no ¬x≤y = y ∷ insert x ys
sort : List A → List A
sort [] = []
sort (x ∷ xs) = insert x (sort xs)
```

Twan van Laarhoven proved complete correctness of various sort algorithms (code gist). We’ll only do a *sortedness* of insertion sort.

Twan uses `_≤?_ : (x y : A) → (x ≤ y ⊎ y ≤ x)`

comparator (`⊎`

is `Either`

), that implies that ordering have to be total. Our `... → Dec (x ≤ y)`

version is less powerful, therefore we’ll need to assume `≤-flip`

.

Twan also defines `insert`

to operate on `Sorted xs`

, our proof is completely external.

There are few auxiliary lemmas, culminating with lemma that `insert`

preserves sortedness, and the theorem that `sort`

produces a sorted list.

One could also show that `sort`

produces a permutation of input list, but that’s something I’m quite confident about already. Note: the proof of that fact won’t need any additional assumptions about `≤`

.

By the way, these proofs show how dependently typed programming is full with various lists.^{1} Luckily (or not) Agda allows reuse of constructor names.

```
module Total (A : Set) (_≤_ : A → A → Set)
(_≤?_ : (x y : A) → Dec (x ≤ y))
(≤-trans : ∀ {x y z} → x ≤ y → y ≤ z → x ≤ z)
(≤-flip : ∀ {x y} → ¬ (x ≤ y) → y ≤ x) -- precise enough!
where
infix 4 _≤*_
open Sort _≤_ _≤?_
data _≤*_ (x : A) : List A → Set where
[] : x ≤* []
_∷_ : ∀ {y ys} → x ≤ y → x ≤* ys → x ≤* y ∷ ys
data Sorted : List A → Set where
[] : Sorted []
_∷_ : ∀ {x xs} → x ≤* xs → Sorted xs → Sorted (x ∷ xs)
≤*-trans : ∀ x y ys → x ≤ y → y ≤* ys → x ≤* ys
≤*-trans x y [] x≤y [] = []
≤*-trans x y (y' ∷ ys) x≤y (y≤y' ∷ y'≤ys) =
≤-trans x≤y y≤y' ∷ (≤*-trans x y ys x≤y y'≤ys)
lem-cons-all≤ : ∀ x y ys → x ≤ y → y ≤* ys → x ≤* y ∷ ys
lem-cons-all≤ x y ys x≤y y≤ys = x≤y ∷ ≤*-trans x y ys x≤y y≤ys
lem-skip : ∀ x y ys → y ≤ x → y ≤* ys → y ≤* insert x ys
lem-skip x y [] y≤x y≤ys = y≤x ∷ y≤ys
lem-skip x y (y' ∷ ys) y≤x (y≤y' ∷ y'≤ys ) with x ≤? y'
... | yes x≤y' = y≤x ∷ y≤y' ∷ y'≤ys
... | no ¬x≤y' = y≤y' ∷ (lem-skip x y ys y≤x y'≤ys)
lem-insert-sorted : ∀ x xs → Sorted xs → Sorted (insert x xs)
lem-insert-sorted x [] s = [] ∷ s
lem-insert-sorted x (y ∷ ys) (y≤ys ∷ sys) with x ≤? y
... | yes x≤y = lem-cons-all≤ x y ys x≤y y≤ys ∷ y≤ys ∷ sys
... | no ¬x≤y = lem-skip x y ys (≤-flip ¬x≤y) y≤ys ∷ sorted-insert-x-ys
where sorted-insert-x-ys = lem-insert-sorted x ys sys
thm-sort-sorted : ∀ xs → Sorted (sort xs)
thm-sort-sorted [] = []
thm-sort-sorted (x ∷ xs) =
lem-insert-sorted x (sort xs) (thm-sort-sorted xs)
```

But what about *partial order*? Wikipedia says following about the topological sort:

In computer science, a topological sort or topological ordering of a directed graph is a linear ordering of its vertices such that for every directed edge from vertex to vertex , comes before in the ordering.

We massage that into simple "there aren’t edges pointing backwards". So instead of saying: "for all sublists `x ∷ ys`

, `x`

is less-than-or-equal than any of `y`

`ys`

" we say "for all subsets `x ∷ ys`

, x is not greater-than any of `y`

`ys`

".

After that, the proof structure is quite similar. I needed to use antisymmetry of `≤`

, which shows that this `Sorted`

predicate won’t hold for *preorder*. I’m not sure whether insertion order would work for preorder, I’m not sure it won’t work either.

```
module Partial (A : Set) (_≤_ : A → A → Set)
(_≤?_ : (x y : A) → Dec (x ≤ y))
(≤-trans : ∀ {x y z} → x ≤ y → y ≤ z → x ≤ z)
(≤-antisym : ∀ {x y} → x ≤ y → y ≤ x → x ≡ y)
where
open Sort _≤_ _≤?_
record _<_ (x y : A) : Set where
constructor le
field
is-le : x ≤ y
not-eq : ¬ (x ≡ y)
open _<_
<-trans₁ : ∀ {x y z} → x < y → y ≤ z → x < z
<-trans₁ (le x≤y ¬x≡y) y≤z = le
(≤-trans x≤y y≤z)
(λ x≡z → ¬x≡y (≤-antisym x≤y (subst (λ i → _ ≤ i) (sym x≡z) y≤z)))
infix 4 _¬>*_
-- x ¬>* ys = x is not larger than any in y ∈ ys
data _¬>*_ (x : A) : List A → Set where
[] : x ¬>* []
_∷_ : ∀ {y ys} → ¬ y < x → x ¬>* ys → x ¬>* y ∷ ys
data Sorted : List A → Set where
[] : Sorted []
_∷_ : ∀ {x xs} → x ¬>* xs → Sorted xs → Sorted (x ∷ xs)
lem-trans-none> : ∀ x y ys → x ≤ y → y ¬>* ys → x ¬>* ys
lem-trans-none> x y [] x≤y [] = []
lem-trans-none> x y (z ∷ zs) x≤y (¬z<y ∷ y≤zs) =
(λ z<y → ¬z<y (<-trans₁ z<y x≤y)) ∷ lem-trans-none> x y zs x≤y y≤zs
lem-flip : ∀ {x y} → x ≤ y → ¬(y < x)
lem-flip {x} {y} x≤y with y ≤? x
... | yes y≤x = λ y<x → not-eq y<x (≤-antisym y≤x x≤y)
... | no ¬y≤x = λ y<x → ¬y≤x (is-le y<x)
lem-cons-none> : ∀ x y ys → x ≤ y → y ¬>* ys → x ¬>* y ∷ ys
lem-cons-none> x y ys x≤y y≤ys =
lem-flip x≤y ∷ lem-trans-none> _ _ _ x≤y y≤ys
lem-skip : ∀ x y ys → ¬ (x ≤ y) → y ¬>* ys → y ¬>* insert x ys
lem-skip x y [] ¬x≤y [] = (λ p → ¬x≤y (is-le p)) ∷ []
lem-skip x y (z ∷ zs) ¬x≤y (¬z<y ∷ y≤zs) with x ≤? z
... | yes x≤z = (λ x<y → ¬x≤y (is-le x<y)) ∷ ¬z<y ∷ y≤zs
... | no ¬x≤z = ¬z<y ∷ (lem-skip x y zs ¬x≤y y≤zs)
lem-insert-sorted : ∀ x xs → Sorted xs → Sorted (insert x xs)
lem-insert-sorted x [] s = [] ∷ s
lem-insert-sorted x (y ∷ ys) (y≤ys ∷ sys) with x ≤? y
... | yes x≤y = lem-cons-none> x y ys x≤y y≤ys ∷ y≤ys ∷ sys
... | no ¬x≤y = lem-skip x y ys ¬x≤y y≤ys ∷ sorted-insert-x-ys
where sorted-insert-x-ys = lem-insert-sorted x ys sys
thm-sort-sorted : ∀ xs → Sorted (sort xs)
thm-sort-sorted [] = []
thm-sort-sorted (x ∷ xs) =
lem-insert-sorted x (sort xs) (thm-sort-sorted xs)
```

Would merge sort work with partial order? I don’t know yet!

Let’s try in Haskell. You *should* try your hypothesis first, before trying to formally prove them. Proving false statements can take a lot of time!

After a little of imports, we’ll defined `isSorted`

check, and try it on a `N5`

lattice. `insertionSort`

works with `leq`

, but a `mergeSort`

only with `<=`

.

```
import Algebra.PartialOrd
import Algebra.Lattice.N5
import Algebra.Lattice.Ordered
import Data.List (sortBy)
import Test.QuickCheck
lt :: PartialOrd a => a -> a -> Bool
lt x y = x /= y && leq x y
notGt :: PartialOrd a => a -> a -> Bool
notGt x y = not (lt y x)
-- | This checks that list is sorted in PartialOrd sense.
isSorted :: PartialOrd a => [a] -> Bool
isSorted [] = True
isSorted (x : ys) = isSorted ys && all (notGt x) ys
-- | Sorted holds when list is sorted using @Ord@
--
-- +++ OK, passed 100 tests.
totalProp :: [Ordered Int] -> Bool
totalProp = isSorted . sortBy compare
-- | Next, let's define insertion sort.
insertionSort :: (a -> a -> Bool) -> [a] -> [a]
insertionSort le = go where
go [] = []
go (x:xs) = insert x (go xs)
insert x [] = [x]
insert x (y : ys) | le x y = x : y : ys
| otherwise = y : insert x ys
-- | And try with a partially ordered set.
--
-- +++ OK, passed 100 tests.
--
-- Works!
m5Prop :: [N5] -> Bool
m5Prop = isSorted . insertionSort leq
-- Then, naive mergesort.
mergeSort :: (a -> a -> Bool) -> [a] -> [a]
mergeSort f = go where
go [] = []
go [x] = [x]
go xs = let (ys, zs) = split xs
in merge (go ys) (go zs)
merge [] ys = ys
merge xs [] = xs
merge (x:xs) (y:ys) | f x y = x : merge xs (y:ys)
| otherwise = y : merge (x:xs) ys
-- | >>> split [1..10]
-- ([1,3,5,7,9],[2,4,6,8,10])
split :: [a] -> ([a],[a])
split [] = ([], [])
split (x:xs) = case split xs of
~(ys,zs) -> (x:zs,ys)
-- | Our 'mergeSort' returns correct results. Works like 'sort'.
--
-- +++ OK, passed 100 tests
mergeProp :: [Int] -> Property
mergeProp xs = mergeSort (<=) xs === sortBy compare xs
-- >>> quickCheck m5Prop2
-- *** Failed! Falsified (after 18 tests and 4 shrinks):
-- [N5b,N5a,N5c]
-- sorted [N5a,N5c,N5b]
--
-- >>> insertionSort leq [N5b,N5a,N5c]
-- [N5c,N5b,N5a]
--
-- Sort is not unique: insertionSort finds an ordering:
-- See picture at https://hackage.haskell.org/package/lattices-2/docs/Algebra-Lattice-N5.html
--
-- >>> leq N5b N5a
-- True
--
-- >>> isSorted [N5c,N5b,N5a]
-- True
--
-- >>> isSorted [N5b,N5a,N5c]
-- True
--
-- >>> isSorted [N5b,N5c,N5a]
-- True
--
m5Prop2 :: [N5] -> Property
m5Prop2 xs =
let xs' = mergeSort leq xs
in counterexample ("sorted " ++ show xs') $ isSorted xs'
```

Note: `Sorted`

is somewhat lax:

But that’s not an issue to me, as I’ll be sorting lists with distinct elements. More on that later.

In Haskell there’s a variety of string types. In Dependent Haskell there will be a variety of various lists and naturals numbers... ... and strings types. https://twitter.com/phadej/status/1147829454187761664↩

This blog post is a thought on a following question: Can we make `cassava`

(= CSV) stuff a bit safer using *fancy types*? This is not a proposal to change `cassava`

, only some ideas and a demonstration of "real world" fancy types (though STLC-implementation also count as real in my world ;). Also a bit of Generics.

This post is not only a Literate Haskell file , but also can be simply run directly with (given a very recent version of `cabal-install`

),

as we specify dependencies. We'll use `fin`

and `vec`

packages. I assume that `data Nat = Z | S Nat`

and `data Vec :: Nat -> Type -> Type`

are familiar to you.^{1}

```
{- cabal:
build-depends:
, base ^>=4.10 || ^>=4.11
, fin ^>=0.1
, vec ^>=0.1.1.1
, tagged ^>=0.8.6
, text ^>=1.2.3.0
ghc-options: -Wall -pgmL markdown-unlit
build-tool-depends: markdown-unlit:markdown-unlit ^>=0.5.0
-}
```

Next a `{-# LANGUAGE Dependent #-}`

collection of extensions...

```
{-# LANGUAGE EmptyCase #-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PartialTypeSignatures #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE StandaloneDeriving #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE UndecidableInstances #-}
{-# OPTIONS_GHC -Wall -Wno-partial-type-signatures -Wno-unused-imports #-}
```

... and imports

```
module Main where
import Control.Monad (forM)
import Data.Bifunctor (first, bimap)
import Data.Kind (Type)
import Data.Tagged (Tagged (..))
import Data.Text (Text)
import Data.Type.Equality ((:~:) (..))
import Data.Fin (Fin (..))
import Data.Type.Nat (Nat (..), SNatI)
import Data.Vec.Lazy (Vec (..))
import GHC.Generics
import Text.Read (readMaybe)
import qualified Data.Fin as F
import qualified Data.Text as T
import qualified Data.Text.IO as T
import qualified Data.Type.Nat as N
import qualified Data.Vec.Lazy as V
```

*Encoding* is often easier than decoding, so let's start with it. Our running example will be programming languages:

```
data PL = PL
{ plName :: Text
, plYear :: Int
, plPerson :: Text
}
deriving (Eq, Ord, Show, Generic)
```

We can create a small database of programming languages we have heard of:

```
pls :: [PL]
pls =
[ PL "Haskell" 1990 "Simon"
, PL "Scala" 2004 "Martin"
, PL "Idris" 2009 "Edwin"
, PL "Perl" 1987 "Larry"
]
```

For encoding, we'll need to be able to encode individual fields / cells. This is similar to what we have in `cassava`

now. I'm *for* using type-classses for such cases, because I want to be able to leverage *Generics*, we'll see that soon.

Thanks to the fancier types, it would be possible to avoid type-classes, still getting something for free, but that's a topic for another post.

```
class ToField a where toField :: a -> Text
instance ToField Text where toField = id
instance ToField Int where toField = T.pack . show
```

As we can serialise individual fields, let us serialise records. We could have a method `toRecord :: r -> [Text]`

as in `cassava`

now, but it is potentially unsafe. Length of the list may vary depending on the record value. So we'd rather use fancy types!

```
-- field count in a record: its size.
type family Size (a :: Type) :: Nat
class ToRecord r where
toRecord :: r -> Vec (Size r) Text
```

It's easy to imagine the `ToRecord PL`

instance. But we rather write a generic implementation for it.

First a generic `Size`

. Recall that `GHC.Generics`

represents records as a binary tree of `:*:`

. To avoid using `Plus`

, i.e. adding up `Nat`

s and concatenating `Vec`

s, we'll `foldr`

like implementation. There is a nested application of `GSizeF`

type family in the `:*:`

-case. GHC wants `UndecidableInstances`

.

```
type GSize a = GSizeF (Rep a) 'Z -- start from zero
type family GSizeF (f :: Type -> Type) (acc :: Nat) :: Nat where
GSizeF U1 acc = acc
GSizeF (K1 i a) acc = 'S acc
GSizeF (M1 i c f) acc = GSizeF f acc
GSizeF (f :*: g) acc = GSizeF f (GSizeF g acc)
```

Using that type family, we can say succintly define:

Also we can check this dependent-language style. If this test fails, it will be a **compilation error**. This definitely blurs the distiction between tests and types :)

Using similar induction on the structure, we can write a generic implementation for `ToRecord`

. Different clauses are handled by different instances of a workhorse class `GToRecord`

.

```
genericToRecord
:: forall r. (Generic r, GToRecord (Rep r))
=> r -> Vec (GSize r) Text
genericToRecord = gtoRecord VNil . from
class GToRecord rep where
gtoRecord :: Vec acc Text -> rep () -> Vec (GSizeF rep acc) Text
instance GToRecord U1 where
gtoRecord xs _ = xs
instance ToField c => GToRecord (K1 i c) where
gtoRecord xs (K1 c) = toField c ::: xs
instance GToRecord f => GToRecord (M1 i c f) where
gtoRecord xs (M1 f) = gtoRecord xs f
instance (GToRecord f, GToRecord g) => GToRecord (f :*: g) where
gtoRecord xs (f :*: g) = gtoRecord (gtoRecord xs g) f
```

The `ToRecord PL`

instance in the user code is a one-liner:

One more thing: column names. Usually CSV files start with a header line. This is where using `Size`

pays off again: Header have to be the same size as content rows. Here I use `Tagged`

to avoid `AllowAmbiguousTypes`

and `Proxy r`

extra argument.

We could write generic implementation for it, but as dealing with metadata in `GHC.Generics`

is not pretty, I'll implement `PL`

instance manually:

The one piece left is an actual `encode`

function. I cut corners by not dealing with escaping for the sake of brevity.

You should notice, that in the implementation of `encode`

we don't care that much about the fact we get `Vec`

s of the same length for each record. `encode`

implementation would work, even with `class ToRecord' r where toRecord' :: r -> [Text]`

. Fancy types are here to help users of a library write correct (by construction) instances.

```
encode :: forall r. (Header r, ToRecord r) => [r] -> Text
encode rs = T.unlines $ map (T.intercalate "," . V.toList)
$ unTagged (header :: Tagged r _)
: map toRecord rs
```

And it works:

```
*Main> T.putStr $ encode pls
name,year,person
Haskell,1990,Simon
Scala,2004,Martin
Idris,2009,Edwin
Perl,1987,Larry
```

Good. Next we'll write an inverse.

Other direction, *decoding* is trickier. Everything could fail. Fields can contain garbage, there might be not enough fields (too much is not such a problem), but most importantly the fields can come in wrong order. Luckily we have fancy types helping us.

Like `ToField`

, `FromField`

is a copy of `cassava`

class:

```
type Error = String
class FromField a where fromField :: Text -> Either String a
instance FromField Text where fromField = Right
instance FromField Int where
fromField t
= maybe (Left $ "Invalid Int: " ++ show t) Right
$ readMaybe $ T.unpack t
```

Also like `ToRecord`

, `FromRecord`

is simple class as well. Note how library users need to deal only with vector of the right size (versus list of any length). We'll also assume that vector is sorted, to match header columns (which could be encoded with fancier types!)

Generic implementation "peels off" provided vector:

```
genericFromRecord
:: forall r. (Generic r, GFromRecord (Rep r))
=> Vec (GSize r) Text -> Either String r
genericFromRecord ts =
let tmp :: Either Error (Rep r (), Vec 'Z Text)
tmp = gfromRecord ts
in to . fst <$> tmp
class GFromRecord rep where
gfromRecord :: Vec (GSizeF rep acc) Text -> Either Error (rep (), Vec acc Text)
instance GFromRecord U1 where
gfromRecord xs = return (U1, xs)
instance FromField c => GFromRecord (K1 i c) where
gfromRecord (x ::: xs) = do
y <- fromField x
return (K1 y, xs)
instance GFromRecord f => GFromRecord (M1 i c f) where
gfromRecord = fmap (first M1) . gfromRecord
instance (GFromRecord f, GFromRecord g) => GFromRecord (f :*: g) where
gfromRecord xs = do
(f, xs') <- gfromRecord xs
(g, xs'') <- gfromRecord xs'
return (f :*: g, xs'')
instance FromRecord PL where
fromRecord = genericFromRecord
```

And a small sanity check:

```
*Main> fromRecord ("Python" ::: "1990" ::: "Guido" ::: VNil) :: Either String PL
Right (PL {plName = "Python", plYear = 1990, plPerson = "Guido"})
*Main> fromRecord ("Lambda Calculus" ::: "in the 1930s" ::: "Alonzo" ::: VNil) :: Either String PL
Left "Invalid Int: \"in the 1930s\""
```

We are now solved all the easy problems. We have set up the public api of library. `To*`

and `From*`

classes use fancy types, we have taken some burden from library users. However the difficult task is still undone: implementing `decode`

.

The example we'll want to work will have extra fields, and the fields shuffled:

```
input :: Text
input = T.unlines
[ "year,name,types,person,website"
, "1987,Perl,no,Larry,https://www.perl.org/"
, "1990,Haskell,nice,Simon,https://www.haskell.org/"
, "2004,Scala,weird,Martin,https://www.scala-lang.org/"
, "2009,Idris,fancy,Edwin,https://www.idris-lang.org/"
]
```

which is

```
*Main> T.putStr input
year,name,types,person,website
1987,Perl,no,Larry,https://www.perl.org/
1990,Haskell,nice,Simon,https://www.haskell.org/
2004,Scala,weird,Martin,https://www.scala-lang.org/
2009,Idris,fancy,Edwin,https://www.idris-lang.org/
```

There's still enough information, we should be able to successfully extract `PL`

.

Zeroth step is to split input into lines, and lines into cells, and extract the header row. That's not the hard part:

```
prepare :: Text -> Either Error ([Text], [[Text]])
prepare i = case map (T.splitOn ",") (T.lines i) of
[] -> Left "No header"
(r:rs) -> Right (r, rs)
```

The hard part is to decode from `[Text]`

into `Vec (Size r) Text`

. And not only we need to decode, but also sort the columns. Our plan would be to

- sort the header row,
*returning the trace of a sort* - use that trace to sort content rows similarly.

We'll require that content rows contain at least as many columns as header row. It's reasonable requirement, and simplifies things a bit. More relaxed requirement would be to require only as much rows as needed, e.g. in our example we could require only four fields, as we aren't interested in the fifth `website`

field.

What's the *trace* of a sort? Technically it's *permutation*. However in this case, it's not regular permutation, as we aren't interested in all fields. It's easier to think backwards, and think which kind of trace would determine the execution in the step 2. We'll be given a `Vec n Text`

for some `n`

, and we'll need to produce a `Vec m Text`

for some other `m`

(= `Size r`

). Let's try to write that as a data type:

```
data Extract :: Nat -> Nat -> Type where
Step :: Fin ('S n) -- take a nth value, x
-> Extract n m -- recursively extract rest, xs
-> Extract ('S n) ('S m) -- cons x xs
Done :: Extract n 'Z -- or we are done.
deriving instance Show (Extract n m)
```

In retrospect, that type is a combination of *less-than-or-equal-to* and *is a permutation* (inductively defined) predicates.^{2}

We can (should!) immediately try this type in action. For what it's worth, the implementations of following functions is quite restricted by their types. There are not much places where you can make a mistake. To be fair, `extract`

and `Extract`

were written simultaneously: `extract`

is structurally recusive in the `Extract`

argument, and `Extract`

has just enough data for `extract`

to make choices.

```
extract :: Extract n m -> Vec n a -> Vec m a
extract Done _ = VNil
extract (Step n e) xs = case delete n xs of
(x, xs') -> x ::: extract e xs'
-- this probably should be in the `vec` library
delete :: Fin ('S n) -> Vec ('S n) a -> (a, Vec n a)
delete FZ (x ::: xs) = (x, xs)
delete (FS FZ) (x ::: y ::: xs) = (y, x ::: xs)
delete (FS n@FS {}) (x ::: xs) = case delete n xs of
(y, ys) -> (y, x ::: ys)
```

For example, given a row and a trace, we can extract fields we want (writing correct trace by hand *is* tricky).

```
*Main> row = "1987" ::: "Perl" ::: "no" ::: "Larry" ::: "https://www.perl.org/" ::: VNil
*Main> trc = Step 1 $ Step F.fin0 $ Step F.fin1 Done :: Extract N.Nat5 N.Nat3
*Main> extract trc row
"Perl" ::: "1987" ::: "Larry" ::: VNil
```

That starts to feel like magic, doesn't it? To complete the whole spell, we need to complete part 1, i.e. construct `Extract`

traces. Luckily, types are there to guide us:

```
columns
:: (Eq a, Show a)
=> Vec m a -- ^ wanted header values
-> Vec n a -- ^ given header values
-> Either Error (Extract n m)
columns VNil _ = Right Done
columns (_ ::: _) VNil = Left "not enought header values"
columns (h ::: hs) xs@(_ ::: _) = do
(n, xs') <- find' h xs -- find first value
rest <- columns hs xs' -- recurse
return $ Step n rest -- record the trace
```

where we use a helper function `find'`

, which finds a value in the `Vec ('S n)`

and returns not only an index, but also a leftover vector. We could write a test:

- if
`Right (n, ys) = find'`

x xs - then
`ys = delete n xs`

```
find'
:: (Eq a, Show a)
=> a
-> Vec ('S n) a
-> Either Error (Fin ('S n), Vec n a)
find' x (y ::: ys)
| x == y = Right (FZ, ys)
| otherwise = case ys of
VNil -> Left $ "Cannot find header value " ++ show x
_ ::: _ -> do
(n, zs) <- find' x ys
return (FS n, y ::: zs)
```

Let's try `columns`

. It takes some time to understand to interpret `Extract`

values. Luckily the machine is there to do that.

```
*Main> columns ("name" ::: "year" ::: VNil) ("name" ::: "year" ::: VNil)
Right (Step 0 (Step 0 Done))
*Main> columns ("name" ::: "year" ::: VNil) ("year" ::: "name" ::: VNil)
Right (Step 1 (Step 0 Done))
*Main> columns ("name" ::: "year" ::: VNil) ("year" ::: "extra" ::: "name" ::: VNil)
Right (Step 2 (Step 0 Done))
*Main> columns ("name" ::: "year" ::: VNil) ("year" ::: "extra" ::: "foo" ::: VNil)
Left "Cannot find header value \"name\""
*Main> columns ("name" ::: "year" ::: VNil) ("name" ::: VNil)
Left "not enought header values"
```

We have three steps

`prepare`

to split input data into header and content rows`columns`

which checks whether there are all fields we want in the provided header, and returns an`Extract`

value saying how to permute content rows.`extract`

which uses an`Extract`

to extract (and order) correct data columns.

We'll use few two functions from `vec`

: `reifyList`

and `fromListPrefix`

.

```
-- Reify any list [a] to Vec n a.
reifyList :: [a] -> (forall n. SNat n => Vec n a -> r) -> r
-- Convert list [a] to Vec n a. Returns Nothing if input list is too short.
fromListPrefix :: SNatI n => [a] -> Maybe (Vec n a)
```

They both convert a list `[a]`

into `Vec n a`

, however they are different

`reifyList`

works for*any*list. As we don't know the length of dynamic inputs,`reifyList`

takes a continuation which accepts`Vec`

of*any*length. That continuation however would know and be able to use the vector length.`fromListPrefix`

tries to convert a list to a vector of*known length*, and thus may fail.

To put it differently, using `reifyList`

we learn the length of the header, and then we require that subsequent content rows are of atleast the same length. Lifting (or promoting) some information to the type level, reduces the amount of dynamic checks we'll need to consequtively e.g. `extract`

doesn't perform any checks.

```
decode :: forall r. (Header r, FromRecord r) => Text -> Either String [r]
decode contents = do
(hs,xss) <- prepare contents
V.reifyList hs $ \hs' -> do
trc <- columns (unTagged (header :: Tagged r _)) hs'
forM xss $ \xs -> do
xs' <- maybe (Left "not enough columns") Right
$ V.fromListPrefix xs
fromRecord (extract trc xs')
```

All done! To convince you that it works, let's run `decode`

on an `input`

we defined at the beginning of this section.

```
main :: IO ()
main = case decode input :: Either String [PL] of
Left err -> putStrLn $ "ERROR: " ++ err
Right xs -> mapM_ print xs
```

```
*Main> main
PL {plName = "Perl", plYear = 1987, plPerson = "Larry"}
PL {plName = "Haskell", plYear = 1990, plPerson = "Simon"}
PL {plName = "Scala", plYear = 2004, plPerson = "Martin"}
PL {plName = "Idris", plYear = 2009, plPerson = "Edwin"}
```

This is a challenging exercise. Improve `decode`

to deal with incomplete data like:

```
input2 :: Text
input2 = T.unlines
[ "year,name,types,person,website"
, "1987,Perl,no,Larry"
, "1990,Haskell,nice,Simon,https://www.haskell.org/"
]
```

Note, the first content row has only four fields so original `decode`

errors with

The goal is to make `decode`

succeed:

```
*Main> mapM_ (mapM print) (decode input2 :: Either String [PL])
PL {plName = "Perl", plYear = 1987, plPerson = "Larry"}
PL {plName = "Haskell", plYear = 1990, plPerson = "Simon"}
```

There are at least two way to solve this. A trickier one, for which there are two hints in footnotes: first^{3} and second^{4}. And a lot simpler way, which "cheats" a little.

There is still a lot places where we can make mistakes. We use `Vec n a`

, so we have `n`

elements to pick. If we instead use heterogenous lists, e.g. `NP`

from `sop-core`

The types would become more precise. We could change our public interface to:

```
type family Fields r :: [Type]
class ToRecord' r where
toRecord' :: r -> NP I (Fields r)
class Header' r where
header' :: Tagged r (NP (K Text) (Fields r))
```

then writing correct versions of `delete`

, `extract`

etc will be even more type-directed. That's is left as an exericise, I suspect that the code shape will be quite the same.

One valid question to ask, is whether row-types would simplify something here. Not really.

For example `vinyl`

's `Rec`

type is essentially the same as `NP`

. Even if there were anonymous records in Haskell, so `toRecord`

could be implemented directly using a built-in function, it would remove only a single problem from many. At it's not much, as `toRecord`

is generically derivable.

In this post I described a complete fancy types usage example, helping us to deal with the untyped real world. Fancy types make library API more precise: we encode (pre/post)conditions like "lists are of the equal length" in the types.

Also we have seen a domain specific"inductive predicate: `Extract`

. It's a library internal, implementation-detail type. Even in "normal" Haskell, not all types (need to) end up into the library's public interface.

The vector example is the *hello world* of dependent types, but here it prevents users from making silly errors, and also make the implementation of a library more robust.

If they aren't, read through e.g. Stitch paper and / or watch a video of a talk Richard Eisenberg presented at ZuriHac '19 (which i saw live, hopefully it will be posted somewhere soon). or older version from NYC Haskell Group meetup (which I didn't watch, only googled for). The definitions in

`fin`

and`vec`

are as follows:↩`data Nat = Z | S Nat data Fin :: Nat -> Type where FZ :: Fin ('S n) FS :: Fin n -> Fin ('S n) data Vec :: Nat -> Type -> Type where VNil :: Vec 'Z a (:::) :: a -> Vec n a -> Vec ('S n) a`

compare

`Extract`

with`LEProof`

and`Permutation`

↩`-- | An evidence of \(n \le m\). /zero+succ/ definition. data LEProof :: Nat -> Nat -> Type where LEZero :: LEProof 'Z m LESucc :: LEProof n m -> LEProof ('S n) ('S m) -- | Permutation. 'PCons' can be interpretted in two ways: -- * uncons head part, insert in a given position in permutted tail -- * delete from given position, cons to permutted tail. data Permutation :: Nat -> Type where PNil :: Permutation 'Z PCons :: Fin ('S n) -> Permutation n -> Permutation ('S n)`

**First hint**implement a function like:`minimise :: SNatI n => Extract n m -> (forall p. SNatI p => Extract p m -> r) -> r -- conservative implementation, not minimising at all minimise e k = k e`

and use it in

`decode`

.↩**Second hint**My variant of`minimise`

uses`LEProof`

and few auxiliary functions:↩`minimise :: Extract n m -> (forall p. N.SNatI p => LEProof p n -> Extract p m -> r) -> r minimiseFin :: Fin ('S n) -> (forall p. N.SNatI p => LEProof p n -> Fin ('S p) -> r) -> r maxLE :: LEProof n p -> LEProof m p -> Either (LEProof n m) (LEProof m n) weakenFin :: LEProof n m -> Fin ('S n) -> Fin ('S m) weakenExtract :: LEProof n m -> Extract n p -> Extract m p`

Since Docker 17.05, there is support for multi-stage builds. This is an example and tutorial for using it to build simple Haskell webapp. The setup is simple: single `Dockerfile`

, yet the resulting docker image is only megabytes large.

Essentially, I read through Best practices for writing Dockerfiles and made a `Dockerfile`

.

*A word of warning:* If you think Nix is the tool to use, I'm fine with that. But there's nothing for you in this tutorial. This is an opinionated setup: Ubuntu and `cabal-install`

's nix-style build. Also all non-Haskell dependencies are assumed to be avaliable for install through `apt-get`

. If that's not true in your case, maybe you should check Nix.

The files are on GitHub: phadej/docker-haskell-example. I refer to files by names, not paste them here.

Assuming you have docker tools installed, there are seven steps to build a docker image:

- Write your application. Any web-app would do. The assumptions are that the app
- listens at port 8000
- doesn't daemonize itself

I use a minimal servant app:

`docker-haskell-example.cabal`

and`Main.hs`

. If you want to learn about servant, its tutorial is a good starting point. Write

`cabal.project`

containing at least`index-state: 2019-06-17T09:52:09Z with-compiler: ghc-8.4.4 packages: .`

- Pinning
`index-state`

makes builds reproducible enough `with-compiler`

select the compiler so it's not the default`ghc`

- Pinning
Add

`.dockerignore`

. The more stuff you can ignore, the better. Less things to copy to docker build context. Less things would invalidate docker cache. Especially hidden files are not hidden from docker, like editors' temporary files. I hide`.git`

directory. If you want to burn git-hash look at known issues section.Add

`Dockerfile`

and`docker.cabal.config`

.`docker.cabal.config`

is used in`Dockerfile`

. In most cases you don't need to edit`Dockerfile`

. You need, if you need some additional system dependencies. The next step will tell, if you need something.Build an image with

If it fails, due missing library, see next section. You'll need to edit

`Dockerfile`

, and iterate until you get a successful build.After successful build, you can run the container locally

This step is important, to test that all runtime dependencies are there.

And try it from another terminal

It should respond something like:

`HTTP/1.1 200 OK Transfer-Encoding: chunked Date: Thu, 04 Jul 2019 16:15:37 GMT Server: Warp/3.2.27 Content-Type: application/json;charset=utf-8 ["hello","world"]`

The `Dockerfile`

is written with a monorepo setup in mind. In other words setup, where you could build different docker images from a single repository. That explains the `--build-arg EXECUTABLE=`

in a `docker build`

command. It's also has some comments explaining *why* particular steps are done.

There are two stages in the `Dockerfile`

, *builder* and *deployment*.

In builder stage we install all **build dependencies**, separating them into different `RUN`

s, so we could avoid cache invalidation as much as possible. A general rule: *Often changing things have to installed latter*.

We install few dependencies from Ubuntu's package repositories. That list is something you'll need to edit once in a while. The assumption is that all non-Haskell stuff comes from there (or some PPA). There's also a corresponding list in *deployment* stage, there we install only non-dev versions.

```
# More dependencies, all the -dev libraries
# - some basic collection of often needed libs
# - also some dev tools
RUN apt-get -yq --no-install-suggests --no-install-recommends install \
build-essential \
ca-certificates \
curl \
git \
libgmp-dev \
liblapack-dev \
liblzma-dev \
libpq-dev \
libyaml-dev \
netbase \
openssh-client \
pkg-config \
zlib1g-dev
```

At some point we reach a point, where we add `*.cabal`

file. This is something you might need to edit as well, if you have multiple cabal files in different directories.

```
# Add a .cabal file to build environment
# - it's enough to build dependencies
COPY *.cabal cabal.project /build/
```

We only add these, so we can build dependencies.

```
# Build package dependencies first
# - beware of https://github.com/haskell/cabal/issues/6106
RUN cabal v2-build -v1 --dependencies-only all
```

and their cache won't be violated by changes in the actual implementation of the webapp. This is common idiom in `Dockerfile`

s. Issue 6106 might be triggered if you vendor some dependencies. In that case change the build command to

listing as many dependencies (e.g. `servant`

, `warp`

) as possible.

After dependencies are built, the rest of the source files are added and the executables are built, stripped, and moved to known location out of `dist-newstyle`

guts.

The deployment image is slick. We pay attention and don't install development dependencies anymore. In other words we install only **runtime dependencies**. E.g. we install `libgmp10`

, not `libgmp-dev`

. I also tend to install `curl`

and some other cli tool to help debugging. In deployment environments where you can shell into the running containers, it helps if there's something you can do. That feature is useful to debug network problems for example.

The resulting image is not the smallest possible, but it's not huge either:

Cold build is slow. Rebuilds are reasonably fast, if you don't touch `.cabal`

or `cabal.project`

files.

If you have

`data-files`

, situation is tricky: Consider using`file-embed-lzma`

or`file-embed`

packages. I.e. avoid`data-files`

.Cabal issue #6106 may require you to edit

`--dependencies-only`

build step, as explained above.Git Hash into built executable. My approach is to ignore whole

`.git`

directory, as it might grow quite large. Maybe uningoring (with`!`

) of`.git/HEAD`

and`.git/refs`

(which are relatively small) will make`gitrev`

and a like work. Please tell me if you try!Caching of Haskell dependencies is very rudimentary. It could be improved largely, if

`/cabal/store`

could be copied out after the build, and in before the build. I don't really know how to that in Docker. Any tips are welcome. For example with`docker run`

one could use volumes, but not with`docker build`

.

Look at the example repository. I hope this is useful for you.

]]>This post is my notes showing that if you apply a modal necessity construction as described by Pfenning and Davies^{1} to linear logic, you get dual intuitionistic linear logic^{2} by Barber. Maybe this is a well known fact, please tell me if it is so! Also we'll connect that to recent developments, where variables have *multiplicities*.

Let us briefly recap necessity/validity from *a judgmental reconstruction of modal logic*^{3}. There are judgments which do not depend on hypotheses about the truth of propositions. Pfenning and Davies introduce the new judgment that A is valid. Evidence for the validity of A is simply unconditional evidence for A.

- If
`∙ ⊢ A true`

then`A valid`

. - If
`A valid`

then`Γ ⊢ A true`

.

After that Pfenning and Davies internalize the judgment as a proposition, we'll follow their notation and write `□A`

for the proposition expressing that A is valid.

At this point we should note that these all makes sense in a linear logic setting. Using the resource-interpretation of linear logic, we can say that if A is valid then to construct A we don't need any (linear) resources: it's an environment-friendly thing.

Pfenning and Davies then separate context into *true* and *valid* parts. We'll do the same, but use Γ for a collection of valid assumptions, and Δ for the collection of true assumptions. (i.e. swapping their notation). At this point it becomes clear that Γ is intuitionistic context, and Δ is linear context. We'll write contexts as `Γ, a : A ; Δ , b : B`

separating context parts by semicolon, and individual assumptions (or parts) by comma. For empty part we'll write `∙`

.

Next we need to adapt the Pfenning and Davies rules for the linear setting. Their system is not linear, so we need to make some adjustments. We'll go through the rules one-by-one.

**Hypothesis / variable / axiom.** We change the `ID`

rule (from my last post, desribing Multiplicative-Additive Intuitionistic Linear Logic) to allow any intuitionistic context. Original `ID`

rule is `a : A ⊢ a : A`

; we additionally allow an arbitrary intuitionistic context Γ (remember: it can be formed from nothing at all).

We'll also need a rule to use variable from an intuionistic context. Using an assumption from intuitionistic context doesn't "consume" anything. Note that linear part is required to be empty, `Δ = ∙`

**Right rule for □.** This rule is very simple, note how the `Δ = ∙`

on both sides of the line.

**Left rule for □.** The correct rule is not immediately obvious, but here it is anyway

```
Γ , a : A ; Δ ⊢ c : C
--------------------------------------------- □L
Γ ; Δ , ba : □A ⊢ case ba of box a -> c : C
```

These rules feels weird. We can show that rules are locally *sound* and *complete*. Local soundness describes a single step of admissibility of cut, kind of β-reduction step. Local completeness describes a way to η-expand. These are simple checks, so we know that right and left rules are well-balanced: not too powerful nor weak. To be fair, I have only seen local soundness and completeness checks done for natural deduction calculi, but I think it makes sense for sequent calculi as well; though we need to invoke `CUT`

for local soundness.

**Local soundness** The way right and left rules are set up, a proof term with right rule on the left, and left rule on the right have to be glued together with `CUT`

. If `CUT`

is admissible (hopefully it is), then we can reduce the term, removing the `CUT`

rule (often pushing it deeper into the term). In sequent calculi `CUT`

rules show where we need to compute.

Assume that we have CUT rules for intuitionistic and linear contexts, `Int-CUT`

and `Lin-CUT`

respectively. If there is `Lin-CUT`

with □R and □L premises:

```
𝓓 𝓔
------------ ----------------
Γ ; ∙ ⊢ A Γ , A ; Δ ⊢ C
------------ □R ---------------- □L
Γ ; ∙ ⊢ □A Γ ; Δ , □A ⊢ C
------------------------------------------------------ Lin-CUT
Γ ; Δ ⊢ let ba = box a in case ba of box a' -> c : C
```

We can admit Cut, reduce the proof, cutting "smaller" things.

```
𝓓 𝓔
----------- ---------------
Γ ; ∙ ⊢ A Γ , A ; Δ ⊢ C
----------------------------------- Int-CUT
Γ ; Δ ⊢ let a' = a in c : C
```

Or more succinctly:

**Local completeness** We can expand

into

```
---------------- Int-ID
Γ , A ; ∙ ⊢ A
---------------- □R
Γ , A ; ∙ ⊢ □A
-------------------------------------------- □L
Γ ; x : □A ⊢ case x of box y -> box y : □A
```

Or more succinctly:

Now if you take a look at the Dual Intuitionistic Linear Logic (1994) report by Andrew Barber, you'll notice that the DILL system is exactly what is described above.

There are two axiom rules, two cut rules, ... The only differences is that necessity is written as !A, not □A.

Of course it's not required to have two separate contexts, but *programming* in DILL is a lot more convenient, than in ILL with only a linear context.

As a small digression, having the rules for □, we can show

where for the last two we need to modify rules for ⊗ and 1 from my last post, but these modifications, as the proofs itself are quite straight-forward.

In this post we saw how simple construction of validity applied to linear logic (ILL) gives DILL. Pfenning and Davies already mention that their □ corresponds to linear ! (pronounced "of course"^{4}).

A (hopefully) interesting exercise is to work through possibility modality, ⋄. Will it give linear ? (pronounced "why not"). I have no idea.

On the other hand, I think that "!A means that A doesn't require any resources" is a nice intuition. If `A = 5€`

, it would be nice to have `!A`

indeed!

There are recent developments in linear logic

- Conor McBride. I got plenty o' nutting' 2016
- Robert Atkey. The syntax and semantics of quantitative type theory. 2018
- Robert Atkey and James Wood Linear metatheory via linear algebra 2019
- Orchard, Liepelt, Eades Quantitative Program Reasoning with Graded Modal Types (Granule) 2019

which instead of having two contexts, use a single context but have an annotation on each assumption. For example, look at the *GrMini* from Granule paper, and syntax in Atkey & Wood extended abstract: they are quite the same, but different from what we just seen. The difference is well illustrated by ⊗R-rule: (our) two context approach rule is

where we split the (linear) context into "structural" parts. Instead we could use a single context, and parametrise it by *usages*, and split the usages, not the context:

Generalising what you use as annotations, you can get Pfenning and Davies’ modal type theory or Barber's DILL, as Atkey and Wood show in their abstract.

To find out how Granule and Atkey & Wood systems differ, it's probably easies to find them on the internet or at conference and discuss directly :)

In hindsight, using annotations/multiplicities/... is an "obvious" discovery. If you try to formalise linear logic fragment in Coq or Agda, *removing* things from the context needs a lot more plumbing than just *marking as used*.

This brief post connects dots from few quite old papers with recent developments. In my opinion, the old stuff is relatively simple, and hopefully helps to understand the new stuff.

Frank Pfenning and Rowan Davies, A judgmental reconstruction of modal logic (2001)↩

Andrew Barber, Dual Intuitionistic Linear Logic (1994)↩

A modal logic is a logic with fancy prefix operators: modalities. I learned about that paper following Proof Theory Foundations -videos by Brigitte Pientka at Oregon Programming Languages Summer School (OPLSS) 2014.↩

It's common knowledge that we can encode ADTs using so called Church-encoding. It's more correctly called Böhm-Berarducci encoding, as they showed how to *systematically* make a translation in a typed setting in 1985.^{1}

or

or

Sometimes these alternative variants are useful: in some mainstream languages there are no sum types, but higher order functions can be at least simulated. Thus: visitor pattern.

However, this blog post is not about visitor pattern, but about linear lambda calculus. The corresponding BB-encodings are easily derived for "normal" types formed with `×`

(pair), `+`

(`Either`

) and recursion. Let's see how similar encodings can be derived for types in linear lambda calculus.

Given a type

we can define a corresponding type

A version Böhm-Berarducci encoding could be

We can curry the `DDict`

record, ending up only term and type applications and abstractions:

`D''`

is the Böhm-Berarducci encoding of `D`

.

More over, we can also define an initial algebra of `D`

:

and we notice that `DDict r`

and `DF r -> r`

are isomorphic. Therefore we get third way to represent `D`

:

For this post we won't think about recursive types. An algebra types of non-recursive types have a phantom argument `r`

, so for non-recursive type `X`

we'll have

Then we can convert `X -> r`

to something simpler. In non-linear setting, we can expand both products and sums to use only functions. Let's see what we can do in linear lambda calculus.

Linear lambda calculus or linear logics come in many flavours. We will be discussing *a higher order intuitionistic linear logic*, with ⊸, ⊗, ⊕, &, 1, 0, ⊤, and ∀ connectives. One can think that ⊗ is a linear version ×, and ⊕ is a linear version of +, but it's not that simple, which I'll try to show in this blog post.

The linear type-system (or logic) rules are presented with *sequents*. I'm lazy, and write them in Unicode-ASCII-diagrams (Haskell highlighting works reasonably well!) instead of LaTeX. The simplest rule is following, saying that if we have a thing in a context, we can use it to conclude our proof/program:

We will write the *proof terms* too, using heavily Haskell-inspired syntax.

Another general rule is `CUT`

, but we won't need it: Cut is admissible. The rest of the rules are *right* and *left sequents* for particular combinators. In right rules the connective is on the right of turnstile (⊢), in the left the connective is on the left, respectively. (I'd like to use ⟹, to highlight that this is a sequent calculus, not a natural deduction; but long arrows often render funny in mono-space fonts/views).

Next we'll need to go through three connectives: ⊗, ⊸ and ∀, before we can meaningfully talk about BB-like-encoding.

The linear times is probably the most simple connective, illustrating the linearity. Its right rule is

To construct a `Pair`

: we split the context into two disjoint parts Γ and Δ, and use Γ to construct the first half of a pair, and Δ to construct the second half of a pair. Thus we cannot write a term for `A ⊢ A ⊗ A`

in linear logic.

The left rule is trickier:

```
Γ; a : A; b : B ⊢ c : C
------------------------------------------------ ⊗L
Γ; ab : A ⊗ B ⊢ case ab of Pair a b -> c : C
```

We `case`

on a `Pair`

to get its halves `a`

and `b`

into the scope, and then continue. We cannot have `fst`

and `snd`

, as those would "use" the whole pair, forgetting other half, violating linearity.

Linear implication uses the premise exactly once. Something you would hope politicians to use more (neither omitting the facts, nor repeating the same things over and over again).

The right rule for linear implication is similar to introduction rule of normal implication

The left rule is again tricky. When we have a linear implication in the context, we can split the rest of the context into two halves. Then we use the other one to deduce the argument to function, and another with new assumption `f a : B`

to deduce the final conclusion.

```
Γ ⊢ a : A Δ ; b : B ⊢ c : C
--------------------------------------- ⊸L
Γ;Δ; f : A ⊸ B ⊢ let b = f a in c : C
```

We can derive another rule:

```
Δ ; b : B ⊢ c : C
--------------------------------------------- ⊸l'
Δ ; a : A; f : A ⊸ B ⊢ let b = f a in c : C
```

which follows by taking `Γ = a : A`

, and discharging the `Γ ⊢ a : A`

using `ID`

-rule. The `⊸L'`

rule cannot be taken as a definition, as using it would often require `CUT`

(which we avoid).

For some reason, universal quantification is often omitted from linear logics. Its rules are similar to as in System F, as in this restricted (non-dependent) settings types are not terms.

The rules involve *substitution* in terms, also `Y`

is assumed to be fresh. They resemble the rules for implication. The right rule says: to construct universally quantified expression, we need to be able to construct it for any variable name (i.e. `X`

is not used in `Γ`

).

Left rule is almost exact copy of ⊸L, except we can make up new type expressions as we go, they won't consume anything in the context. `T type`

is a pedantic addition to remind us about that, but we'll omit it for brevity.

```
T type Γ; b' : B [T/X] ⊢ c : C
-------------------------------------- ∀L
Γ; b : ∀X. B ⊢ let b' = b @T in c : C
```

Now we have rules and syntax for times, implication and universal quantification, so we can BB-encode times-connective next.

It might be easy to guess the BB-encoding of *times*:

That's a reasonable guess if we can show `A ⊗ B ⊸ R ≡ A ⊸ B ⊸ R`

isomorphism.

Here, the `LHS ≡ RHS`

notation means `LHS ⊢ RHS`

and `RHS ⊢ LHS`

. The ⇒ direction is quite direct (read bottom-up):

```
------- ID
R ⊢ R
--------------- ⊸L'
B ; B ⊸ R ⊢ R
----------------------- ⊸L'
A ; B ; A ⊸ B ⊸ R ⊢ R
----------------------- ⊗L
A ⊗ B ; A ⊸ B ⊸ R ⊢ R
------------------------- ⊸R
A ⊗ B ⊢ (A ⊸ B ⊸ R) ⊸ R
----------------------------- ∀R
A ⊗ B ⊢ ∀R. (A ⊸ B ⊸ R) ⊸ R
```

Terms were omitted for clarity, but the last line would look like (after in lining `let`

s).

The reverse, ⇐ direction is quite direct too (~all proofs in this fragment of linear logic are quite direct, that's the nice thing about whole thing)

```
...
--------------- ⊗R
A ; B ⊢ A ⊗ B
------------------- ⊸R² --------------- ID
⊢ (A ⊸ B ⊸ A ⊗ B) A ⊗ B ⊢ A ⊗ B
------------------------------------------ ⊸L
(A ⊸ B ⊸ A ⊗ B) ⊸ A ⊗ B ⊢ A ⊗ B
--------------------------------- ∀L
∀R. (A ⊸ B ⊸ R) ⊸ R ⊢ A ⊗ B
```

The final line after inlining is:

These are about what you would expect in non-linear calculus for × and →. The derivation trees would be similar. However with ⊕ (and &) things get more interesting.

The plus connective looks very much like + in non-linear logic. I'd say, there shouldn't be anything surprising in the rules

```
Γ ⊢ a : A
------------------- ⊕R₁
Γ ⊢ InL a : A ⊕ B
Γ ⊢ b : B
------------------- ⊕R₂
Γ ⊢ InR b : A ⊕ B
```

```
Γ ; a : A ⊢ c₁ : C Γ ; b : B ⊢ c₂ : C
--------------------------------------------------------------- ⊕L
Γ ; ab : A ⊕ B ⊢ case ab of { InL a -> c₁ ; InR b -> c₂ } : C
```

At this point, one could think that we could do

analogously to `either`

. But that doesn't work: Try! The right hand side type says that we have to use both: (A ⊸ R) and (B ⊸ R) arguments, yet we'll need only either one. We'll need a &-connective.

With, & is an interesting construction. Its rules are mirrored versions of ⊕ rules, on the other hand they look like rules for ×.

Right rule is very similar to ⊗R, but note that both halves are constructed using the same context:

The above works as left rules only allow to use either half of the with-pair. We can choose, but cannot use both; in ⊗L-rule we cannot choose, must use both.

```
Γ ; a : A ⊢ c : C
------------------------------------------- &L₁
Γ ; ab : A & B ⊢ let a = fst ab in c : C
Γ ; b : B ⊢ c : C
------------------------------------------- &L₂
Γ ; ab : A & B ⊢ let b = snd ab in c : C
```

Using *with* we can BB-encode *plus*.

That's using

distributivity property.

Instead of derivation tree, I only write the proof terms. The ⇒ direction:

```
lhs : A ⊕ B
⊢ Λ R -> λ f -> case lhs of
InL a -> fst f a
InR b -> snd f b
: ∀R. ((A ⊸ R) & (B ⊸ R)) ⊸ R
```

Depending on a `lhs`

branch we pick proper half of with argument.

The reverse, ⇐ direction

So let's try to write a BB-encoding for A & B. Reasonable guess is

The ⇒ direction works out:

```
lhs : A & B
⊢ Λ R -> λ arbr -> case arbr of
InL ar -> ar (fst lhs)
InR br -> br (snd lhs)
: ∀R. ((A ⊸ R) ⊕ (B ⊸ R)) ⊸ R
```

However the opposite direction doesn't:

where ⊢ ??? : (A ⊸ A & B) ⊕ (B ⊸ A & B). If we knew the future, we could put the correct `InL`

or `InR`

argument for `???`

, to match the later happening `fst`

or `snd`

. But we don't know how the result will be used.

We have to give up.

If we lookup in Girard's Linear Logic we'll learn that there are following properties

but the third one is only half-distributive, i.e. works only in one direction.

Unfortunate, but known fact. So, we cannot (at least obviously) simulate A & B using something else. Maybe *with* is an essential connective, you simply cannot admit. In other words, if you want something like *with*, you need to add that.

One way to understand linear logic is the resource-interpretation. If we set A = 2€, B = 3$, R = soda, then

`A & B ⊸ R`

means "we put both 2€ and 3$ into a machine, it will make a choice (not using other currency) and give use a soda back"`(A ⊸ R) ⊕ (B ⊸ R)`

means "we make a choice, whether to put in 2€ and get a soda, or to put in 3$ and get a soda"

This example is especially relevant when paying with a credit card. You may pay in a local currency or your currency. If the exchange rate is fair, i.e. A ≈ B, then the two approaches are essentially the same. But when A ≉ B, maybe it's good to have a choice.

Note, that ⇒ direction is a machine asking you to make a choice (often they do), but it could make choice for you (charge local currency if you use contactless; or better option for merchant). But if you use cash, making a choice which currency to use, you cannot be charged price in different one.

Story with units follows the lines of related binary-connectives.

One, 1 is unit of times. One doesn't carry any information, so we can freely introduce and remove it from the context. t has expected rules:

Zero, 0 is unit of plus. It has only a left rule, if there's a value of type 0 in a context, "all is lost", anything is derivable.

Top, ⊤ is unit of with. It has only a right rule. It's an interesting type which let us dismiss stuff, putting "boring" stuff a side. But we cannot ever get completely rid of of them, neither extract back. A bit like black hole, I guess.

Corresponding BB-encodings are

Conversions: an exercise.

The ⇒ direction is given by `absurd`

. The ⇐ by instantiating with `@0`

.

⊤, as &, cannot really be admitted. We can squash everything into ⊤, but it's impossible to get rid of it.

In higher order linear lambda calculus, type and term abstraction and application is not enough. The plus & and top ⊤ stand out.

On the other hand, if your linear language doesn't have plus and top, there are still a lot of you can do. Then the setup is close to normal lambda calculus; yet subtly different, you must be aware that not all tricks will work out.

This post is a not-so-gentle introduction into linear lambda calculus. I plan to write-mumble more, so this post will act as a reference point.

```
Γ ⊢ a : A Δ ; b : B ⊢ c : C
--------------------------------------- ⊸L
Γ;Δ; f : A ⊸ B ⊢ let b = f a in c : C
```

```
Γ; a : A; b : B ⊢ c : C
------------------------------------------------ ⊗L
Γ; ab : A ⊗ B ⊢ case ab of Pair a b -> c : C
```

```
T type Γ; b' : B [T/X] ⊢ c : C
-------------------------------------- ∀L
Γ; b : ∀X. B ⊢ let b' = b @T in c : C
```

```
Γ ⊢ a : A
------------------- ⊕R₁
Γ ⊢ InL a : A ⊕ B
Γ ⊢ b : B
------------------- ⊕R₂
Γ ⊢ InR b : A ⊕ B
```

```
Γ ; a : A ⊢ c₁ : C Γ ; b : B ⊢ c₂ : C
--------------------------------------------------------------- ⊕L
Γ ; ab : A ⊕ B ⊢ case ab of { InL a -> c₁ ; InR b -> c₂ } : C
```

```
Γ ; a : A ⊢ c : C
------------------------------------------- &L₁
Γ ; ab : A & B ⊢ let a = fst ab in c : C
Γ ; b : B ⊢ c : C
------------------------------------------- &L₂
Γ ; ab : A & B ⊢ let b = snd ab in c : C
```

Supporting wide (version) ranges of dependencies is a common problem in software engineering. In particular, supporting many major GHC versions is sometimes tricky. In my opinion it's not because Haskell-the-language changes, very few extensions are essential for library-writing^{1}. A tricky part is the changes in the so called boot libraries: `base`

, `transformers`

... Luckily, there is a collection of *compatibility packages*, which smooth the cross-GHC development experience.

Compatibility packages are symptom of a greater problems: non-reinstallable `ghc`

(and `base`

). That will hopefully change soon: reinstallable `lib:ghc`

is mentioned on GHC-8.8.1 page in a planned section.

Another, somewhat related problem is orphan instances. Sometimes instances are just forgotten, and as upgrading a proper home package is tricky one, a module with orphan instances is the best one can do. Or maybe the reason is as simple as that neither type-class nor data-type defining package can^{2} depend on other.

As some of these compatibility packages define orphan instances, it would be good that people used these packages instead of defining their own orphans. They are easy to update, if something is still missing.^{3}

In this post I'll discuss some compatibility packages, I'm aware of. I'll also mention few extras.

Acknowledgments: Ryan Scott maintains a lot of packages listed below. I cannot appreciate enough all the time and work put into their maintenance.

`base`

is a core package. Along the years, there were bigger changes like Applicative-Monad-Proposal, Semigroup-Monoid-Proposal, MonadFail-Proposal, but also small changes, like addition of new instances and functions.

The scope of `base-compat`

is to provide functions available in later versions of base to a wider (older) range of compilers.

One common at the time problem was `<$>`

not in scope, due AMP. This kind of small problems are easily fixed with

`base-compat`

provides "an alternative" prelude, which I can recommend to use.

Similarly to `<$>`

, `base-compat`

provides `Semigroup`

right out of `Prelude.Compat`

. However due the fact that `base-compat`

does not add any orphan instances (`base-orphans`

does). neither backport any types, `Semigroup`

is provided only with GHC-8.0 / `base-4.9`

. Latter is solved by `base-compat-batteries`

.

Also `base-compat`

(and `base-compat-batteries`

) provide a lot of `.Compat`

modules

`base-compat-batteries`

provides the same API as the `base-compat`

library, but depends on compatibility packages (such as `semigroups`

) to offer a wider (or/and more complete) support than `base-compat`

.

The most is understood by looking at `build-depends`

definitions of `base-compat-batteries`

:

```
if !impl(ghc >= 7.8)
build-depends:
tagged >= 0.8.5 && < 0.9
if !impl(ghc >= 7.10)
build-depends:
nats >= 1.1.2 && < 1.2,
void >= 0.7.2 && < 0.8
if !impl(ghc >= 8.0)
build-depends:
fail >= 4.9.0.0 && < 4.10,
semigroups >= 0.18.4 && < 0.20,
transformers >= 0.2 && < 0.6,
transformers-compat >= 0.6 && < 0.7
if !impl(ghc >= 8.2)
build-depends:
bifunctors >= 5.5.2 && < 5.6
if !impl(ghc >= 8.6)
build-depends:
contravariant >= 1.5 && < 1.6
```

In some cases (like application development), `base-compat-batteries`

is a good choice to build your own prelude on. Even some projects are often built with single GHC only, `base-compat-batteries`

can help smooth GHC migrations story. You can adapt to the newer `base`

without updating the compiler (and all other bundled dependencies).

However when developing libraries, you might want be more precise. Then you can use the same conditional `build-depends`

definitions to incur dependencies only when `base`

is lacking some modules. Note that we use GHC version as a proxy for `base`

version^{4}

`bifunctors`

provides`Data.Bifunctor`

,`Data.Bifoldable`

and`Data.Bitraversable`

`contravariant`

provides`Data.Functor.Contravariant`

.`fail`

provides`Control.Monad.Fail`

`nats`

provides`Numeric.Natural`

`semigroups`

provides`Data.Semigroup`

`tagged`

provides`Data.Proxy`

`void`

provides`Data.Void`

module.

Note, that some of this packages provide additional functionality, for example `semigroups`

have `Data.Semigroup.Generic`

and `contravariant`

`Data.Functor.Contravariant.Divisible`

modules.

`base-orphans`

defines orphan instances that mimic instances available in later versions of base to a wider (older) range of compilers.

My personal favourite is

instance, which is in `base`

only since `4.7.0.0`

.

A word of warning: sometimes the instance definition changes, and that cannot be adopted in a library.

`generic-deriving`

is the package providing `GHC.Generics`

things.

Notably it provides missing `Generic`

instances for things in `base`

. If you ever will need `Generic (Down a)`

, it's there.

`transformers`

is a well known library for monad transformers (and functors!).

`transformers-compat`

backports versions of types from newer transformers. For example an `ExceptT`

transformer.

Note that for example `Data.Functor.Identity`

may be in `base`

, `transformers`

or `transformers-compat`

, so when you do

it may come from different packages.

`writer-cps-transformers`

have become a compatibility package, as since `0.5.6.0`

`transformers`

itself provide "stricter" `Writer`

monad in `Control.Monad.Trans.Writer.CPS`

.

There is also a `writer-cps-mtl`

package which provides `mtl`

instances.

`deepseq`

provides methods for fully evaluating data structures.

Since `deepseq-1.4.0.0`

(bundled with GHC-7.10) the default implementation of `rnf`

uses Generics. Before that

worked for any type and did force only to WHNF: `rnf x = x`

seq`()`

. If your library define types, and support older than GHC-7.10 compilers, the correct variant is to define `rnf`

explicitly, using `deepseq-generics`

*Template Haskell* (TH) is the standard framework for doing type-safe, compile-time meta programming in the GHC. There are at least two compat issues with Template Haskell: defining `Lift`

instances, missing `Lift`

instances, and changes in `template-haskell`

library itself.

They are solved by three (or four) packages:

`th-lift`

provides`TemplateHaskell`

based deriving code for`Lift`

type-class (and defines`Lift Name`

);`th-lift-instances`

provides instances for small set of core packages, and acts as a instance compat module (e.g. provides`Lift ()`

, which wasn't in`template-haskell`

from the beginning);`th-orphans`

provides instances for the types in`template-haskell`

- and
`th-abstraction`

which helps write Template Haskell code.

Note: `th-lift`

and `th-lift-instances`

only use `TemplateHaskellQuotes`

, therefore they don't need interpreter support. That means that your library can provide Template Haskell functionality without itself requiring it. This is important e.g. for GHCJS (template haskell is *very* slow), or in cross-compilation (tricky issues), or the simple fact the system doesn't have dynamic loading (See *Dyn Libs* in GHC supported platforms)^{5}.

`Lift`

type-class provides `lift`

method, which let's you *lift* expressions in Template Haskell quotations. It's useful to embed data into final library

`th-lift`

provides `TemplateHaskell`

based deriving code for `Lift`

type-class.

With GHC-8.0 and later, you can write

however, for older GHCs you can use `th-lift`

```
{-# LANGUAGE TemplateHaskell #-}
import Language.Haskell.TH.Lift (deriveLift)
data MyType = ...
deriveLift ''MyType
```

`th-lift-instances`

provides instances for small set of core packages.

to get e.g. `Lift Text`

instance.

`th-orphans`

provides instances for `template-haskell`

types, in particular `Ord`

and `Lift`

. This package is useful when you *write* Template Haskell code, not so when you use it.

`th-abstraction`

is not precisely a compat package, but it normalizes variations in the interface for inspecting datatype information via Template Haskell so that packages can use a single, easier to use informational datatype while supporting many versions of Template Haskell.

If you can write your TH code using `th-abstraction`

interface, it's way simpler than all CPP involved with raw `template-haskell`

usage.

`binary`

provides binary serialisation. There are various alternatives^{6}, but `binary`

is bundled with GHC, so if you don't have special requirements it's good enough default choice. The benefit of `binary`

is that there **is** a lot of support, many packages provide `Binary`

instances for their types.

`binary-orphans`

provides instances defined in later versions of `binary`

package. For example it provides `MonadFail Get`

instance, which was the main motivation for the creation of the package.

`binary-instances`

scope is broader, it provides `Binary`

instances for types in some popular packages: `time`

, `vector`

, `aeson`

et cetera.

`bytestring`

provides an immutable byte string type (pinned memory).

`bytestring-builder`

provides `Data.ByteString.Builder`

and `Data.ByteString.Short`

modules for old `bytestring`

.

The commonly needed missing piece is `Data.ByteString.Lazy.toStrict`

and `fromStrict`

. They are not needed that often though, so I have written it inline when needed. Compat `toStrict`

is actually is more efficient than it looks^{7}:

```
import qualified Data.ByteString as BS
import qualified Data.ByteString as LBS
fromStrict :: BS.ByteString -> LBS.ByteString
toStrict :: LBS.ByteString -> BS.ByteString
#if MIN_VERSION_bytestring(0,10,0)
fromStrict = LBS.fromStrict
toStrict = LBS.toStrict
#else
fromStrict bs = LBS.fromChunks [bs]
toStrict lbs = BS.concat LBS.toChunks lbs -- good enough.
#endif
```

`time`

provides most time related functionality you need. In `1.9`

version it got a lot of nice things, including `CalendarDiffDays`

to represent "calendar" day difference.

`time-compat`

shims the `time`

package to it's current latest version. This is my recent experiment. *All* `time`

modules have a `.Compat`

version. This means that you can change to dependency definition

and imports

and get reasonably^{8} compatible behaviour across different GHC (7.0 ... 8.8) and time (1.2 ... 1.9.2) versions.

We already mention `th-lift`

which provide *Template Haskell* based functionality to derive `Lift`

type-class instances. There are other classes to be derived.

`deriving-compat`

provides Template Haskell functions that mimic deriving extensions that were introduced or modified in recent versions of GHC.

Particularly, it provides a way to mimic `DerivingVia`

which is only in GHC-8.6+. The `deriving-compat`

is ugly, but if you are stuck with GHC-8.2 or GHC-8.4, that's an improvement: you can prepare code to be `DerivingVia`

ready.

So if real `DerivingVia`

looks like

```
{-# LANGUAGE DerivingVia #-}
data V2 a = V2 a a
deriving (Functor)
deriving (Semigroup, Monoid) via (Ap V2 a)
instance Applicative V2 where ...
```

the `deriving-compat`

way looks like

```
{-# LANGUAGE TemplateHaskell, ... #-}
import Data.Deriving.Via
data V2 a = V2 a a
deriving (Functor)
instance Applicative V2 where ...
deriveVia [t| forall a. Semigroup a => Semigroup (V2 a) `Via` (Ap V2 a) |]
deriveVia [t| forall a. Monoid a => Monoid (V2 a) `Via` (Ap V2 a) |]
```

It feels like `StandaloneDeriving`

with all pros and cons of it.

`QuickCheck`

is a well known library for random testing of program properties. It's usage relies on `Arbitrary`

instances for various types.

The goal of `quickcheck-instances`

is to supply `QuickCheck`

instances for types provided by the Haskell Platform. Also to keep `QuickCheck`

dependency light, and also CPP free, `quickcheck-instances`

provides instances for types like `Natural`

or `NonEmpty`

.

[`hashable](https://hackage.haskell.org/package/hashable) defines a class,`

Hashable`, for types that can be converted to a hash value.

`hashable-time`

is a small package providing `Hashable`

instances for types from `time`

.

If you want to use `unordered-containers`

with `time`

types

to get missing instances.

I have to mention one personal mistake: `aeson-compat`

. `aeson`

is not a GHC bundled boot package, and upgrading its version shouldn't be a problem. Creating `aeson-compat`

felt like a good idea back then, but currently I'd recommend to just use as recent `aeson`

as you need. I'd also advice against creating any compatibility packages for any other non-boot libraries, it's quite pointless.

In general, I don't see point sticking to the too old versions of non-boot dependencies. Virtually no-one uses the low ends of dependency ranges, at least based by amount of incorrect lower-bounds I find during my own experiments^{9}. *Stackage* and *nixpkgs* add some friction into the equation. Allowing versions of dependencies in a current *Stackage LTS* is a kind thing to do, but if you need newer version, then at least I don't feel bad myself about putting higher lower bound.

It's quite likely that `text`

will adopt UTF-8 as the internal representation of `Text`

. My gut feeling says that the change won't be huge issue for the most of the users. But for example for the `aeson`

it will be: it would be silly not to exploit UTF-8 representation. `attoparsec`

should need adaptation as well. However, `text`

is bundled with GHC, and though it shouldn't be a problem to upgrade (as only `mtl`

, `parsec`

and `Cabal`

depend on it, not `ghc`

itself), it's hard to predict what subtle problems there might be. `text-utf8`

has an *old* fork of `aeson`

, but it pretends there's only new `text-utf8`

. So there is a lot of (compatibility) work to do for someone who cares about the details!

Library features are rarely the problem preventing wide GHC support windows. Unfortunately the compatibility story grew up organically, so the naming is inconsistent, which hurts discoverability of these compatibility packages. However, I think it's very great, given it's somewhat uncoordinated and voluntary effort. It's possible to write code which works for at least GHC 7.4 ... GHC-8.6. GHC-7.4.1 was released in 2012, seven years ago. We should measure support windows in years, not major versions. And from this point-of-view Haskell is definitely suitable for an enterprise development!

Libraries will keep changing, and it's good to think about compatibility a bit too. Luckily, there is prior art in Haskell eco-system to learn from!

Using

`BlockArguments`

or similar syntax-only-extension, if completely fine. But I don't think using them is alone a good reason to not supporting older GHCs. Also, I don't think that*dropping*support for older GHC support only because it's old, is not a good enough reason.↩Of course they can, but in addition to being social problem, it's also technical challenge: Completely linearising dependency graph into a chain is non desirable, though

*data-type*package depend on*type-class*packages may still offer enough slack in the dependency graph.↩There is an opinion that "lawless" classes like

`Arbitrary`

,`Binary`

,`FromJSON`

should not exist, but rather we should use explicit values. I think that opinion is wrong, at the very least too extreme. Having explicit values and combinators is good, we can have both. These kind of type-classes enable generic programming and things like`servant`

: they let compiler write code for you. But this topic is better discussed in a separate blog post.↩Semantically using GHC version as proxy for

`base`

is imprecise, but it's simpler than currently available alternatives.↩In my opinion Template Haskell,

`build-type: Custom`

`Setup.hs`

scripts,`unsafeCoerce`

or`unsafePerformIO`

are all in the same group. Every time you use any, you have to write 200 words essay. Any consequtive use will require a 100 words longer one. Please send me those essays, I promise to publish the best parts.↩Check

`serialise`

which is also fast, but also uses CBOR format. That is useful, as the binary representation is self-descriptive, at least to some degree.↩Compare

`toStrict`

from 0.10.8.2 with`concat`

from 0.9.2.1.↩For example changing instance implementation is impossible, therefore parsing and formatting behaves as in underlying

`time`

versions. In any case, more compatible then bare`Data.Time`

.↩I actually compile a lot packages sweeping their entire dependency-range. My

`~/cabal/.store`

have grown to 175G in a past month, and it contains e.g. 20 variants of`text-1.2.3.1`

for a single compiler version. Often lower bounds have bit-rotten, but sometimes there are also "interesting" interactions in the middle of dependency ranges.↩

This weekend, I saw a tweet about formatting tabular data. A complaint that it takes two hundred or so lines of C-code for something which is a oneliner.

Well, a solution (in Haskell) was a one-liner, *it fits in a tweet* But in my opinion, it wasn't beautiful nor cool.

The solution below also fits in a tweet:

```
tabular zs = unlines rs where
(cs, ws, rs) = go zs
go [] = (0, repeat 0, [])
go (x : xs) =
let (c, w, ys) = go xs
in (max c (length x), zipWith max w (map length x ++ repeat 0),
unwords (take cs (zipWith fl x ws)) : ys)
fl s n = s ++ replicate (n - length s) ' '
```

**EDIT 2018-05-12:** Joseph C. Sible kindly pointed out that we can make it even more concise using `foldr`

(258 characters):

```
tabular zs = unlines rs where
(cs, ws, rs) = foldr go (0, repeat 0, []) zs
go x (c, w, ys) =
(max c (length x), zipWith max w (map length x ++ repeat 0),
unwords (take cs (zipWith fl x ws)) : ys)
fl s n = s ++ replicate (n - length s) ' '
```

It's also beyond mortal human comprehension. And not only because all symbol names are short. It's special for another reason too. It cannot be re-written as is, in say JavaScript or Scala, as it relies on **laziness**.

It's **cool**.

Not because it uses laziness.

Because, thanks to laziness, we make **only a single pass** through the table rows.

During that pass we calculate the number of columns, maximum column widths, and *simultaneously* (!!!) take the final amount of columns padded to their final widths!

When you know what happens, the code is hopefully more understandable; maybe even obvious. You still need to convince yourself that constructed computational *graph* (not *tree*) doesn't contain cycles and can be reduced into final result (i.e. won't loop). (I had to write it down, and run it, to convince myself).

One could go further and make only single pass per table row, but I'll leave that as an exercise for a reader. :)

It's crazy what Haskell does to you. When you learn to use its superpowers, there are no way back.

Finally, the less golfed version:

```
module Table where
table :: [[String]] -> String
table cells = unlines rows
where
cols :: Int
rowWidths :: [Int]
rows :: [String]
(cols, rowWidths, rows) = foldr go (0, repeat 0, []) cells
go :: [String] -> (Int, [Int], [String]) -> (Int, [Int], [String])
go xs (c, w, yss) =
( max c (length xs)
, zipWith max w (map length xs ++ repeat 0)
, unwords (take cols (zipWith fill xs rowWidths))
: yss
)
fill :: String -> Int -> String
fill s n = s ++ replicate (n - length s) ' '
```

It works. Take an "unformatted" table of strings:

and format it into a nice table:

]]>This posts complements two other recent blogs: *Code smell: Boolean blindness* by Thomas Tuegel and *Ceci n'est pas un default* by Guillaume Allais.

Thomas writes

The popular term “boolean blindness” refers to the information lost by functions that operate on Bool when richer structures are available

and indeed, in one project I wrote a function with a very horrible type:

I could add comments, but those are unchecked by a compiler. I could use `Maybe`

or some other richer type, but these `Bool`

s are coming from command line arguments, which are just single bit of information. What else can I do?

Phantom types is one possible solution.

After small module header...

```
{-# LANGUAGE KindSignatures, DataKinds #-}
{-# LANGUAGE FunctionalDependencies, FlexibleContexts #-}
{-# LANGUAGE ScopedTypeVariables, DeriveAnyClass #-}
-- | Because 'Flag' constructor isn't exposed,
-- we have to explicitly 'toFlag' and 'fromFlag'.
-- That way it's less likely we mix up bare Booleans.
module Flag (
-- * Flag
Flag, toFlag, fromFlag,
-- * HasDefault
HasDefault,
-- * optparse-applicative
showHide,
switchA,
switchB,
) where
import Control.Applicative ((<|>))
import Data.Proxy (Proxy (..))
import Data.Semigroup (Semigroup (..))
import Data.Singletons.Bool
import qualified Options.Applicative as O
```

... we can define `Flag`

:

```
newtype Flag t = MkFlag Bool
toFlag :: t -> Bool -> Flag t
toFlag _ = MkFlag
fromFlag :: t -> Flag t -> Bool
fromFlag _ (MkFlag b) = b
```

A `Flag`

is just a `Bool`

. But `Flag Foo`

and `Flag Bar`

are different types!

As the module export list already revealed, the `MkFlag`

constructor is not exported. The only way to construct and match on `Flag`

is to use `toFlag`

and `fromFlag`

functions. Both functions take an additional `t`

argument. The idea is to define a data type per flag and use the single value as a proxy for a type.

```
data ShowBuiltin = ShowBuiltin
data ShowGlobal = ShowGlobal
data TopoReverse = TopoReverse
data ShowFlags = ShowFlags
```

We define a nullary constructor, so the type can act as its own `Proxy`

. Using `Flag`

, we can make `doTopo`

signature much more comprehensible:

```
doTopo
:: UseColors
-> Flag ShowBuiltin
-> Flag ShowGlobal
-> PlanJson
-> Flag TopoReverse
-> Flag ShowFlags
-> IO ()
doTopo ... =
-- here the redundancy is good, it will help catch errors
if fromFlag ShowBuiltin showBuiltin
then ...
else ...
```

Now the type signature is much more self-explanatory, and it's impossible (or at least quite hard) to mixup different flags. And note, this is completely `Haskell98`

solution. So far we didn't need any extensions.

Another alternative would been to define a helper structure,

```
data TopoOpts = TopoOpts
{ toUseColors :: UseColors
, toShowBuiltin :: Bool
, toShowGlobal :: Bool
, toPlanJson :: PlanJson
, toTopoReverse :: Bool
, toShowFlags :: Bool
}
```

especially with `RecordWildCards`

language extension, it's convinient to use. Also with this approach it's harder to mixup different `Bool`

s. Both `Flag`

and a **record** name Booleans.

For whoever is wondering, the `UseColors`

type is a three way setting: `always`

, `never`

or `auto`

, so I wrote a special type for it. One could write own type for each Boolean flag too. I didn't explore that alternative. We'd need to `case`

, as there aren't natural candidate to proxy in `fromFlag`

variant (which can be made with `Generics`

: but one have to be careful to define values in order as in `data Bool = False | True`

!).

It depends which approach, `Flag`

or a record, is more convinient (or even both together!). `Flag`

approach let us do some nice extra stuff, thanks to a type-level tag. But we have to departure the Haskell98-land. Inspired by Guillaume's post, we can do a little of dependent-ish programming. I use my `singleton-bool`

library here.

First we define `HasDefault`

class.

```
-- | Has default boolean value.
class SBoolI def => HasDefault (def :: Bool) t | t -> def
-- | Flag with a default value.
def :: forall t def. HasDefault def t => t -> Flag t
def t = toFlag t (reflectBool (Proxy :: Proxy def))
```

The `FunctionalDependencies`

(and not `TypeFamilies`

) are used so we can use `DeriveAnyClass`

extension:

Now the phantom type isn't really-really phantom anymore, `HasDefault`

constraint makes it "relevant".

We can use `HasDefault`

to implicitly wire-in things, for example when making `optparse-applicative`

parsers. Remember, `Flag`

s often come from command line arguments.

I'd like to use specialised parser for `Show*`

flags. Now we can use the fact that we know the default value: And the default value is specified close to flag declaration (visible in Haddocks!), and not hidden somewhere in the command line parser code. That's a win. So we can write quite generic `showHide`

parser combinator:

```
showHide :: HasDefault def t => t -> String -> String -> O.Parser (Flag t)
showHide t n d =
O.flag' (toFlag t True) (O.long ("show-" ++ n) <> O.help d)
<|> O.flag' (toFlag t False) (O.long ("hide-" ++ n))
<|> pure (def t)
```

*Note:* we can go even further: we could define a `HasParser`

class and with a little `DerivingVia`

magic parsers could be completely derivable. I leave that as an exercise :)

Another use case: super power `switch`

. We can make GHC verify that the default value is `False`

, so if we decide to change the default, we won't forget to fix the cli parsers:

```
switchA :: HasDefault 'False t => t -> String -> String -> O.Parser (Flag t)
switchA t n d = fmap (toFlag t) $ O.switch $ O.long n <> O.help d
```

Or we can make switch flip automatically:

```
switchB :: HasDefault def t => t -> String -> String -> O.Parser (Flag t)
switchB t n d = fmap (toFlag t) $ O.flag v (not v) $ O.long n' <> O.help d
where
n' = if v then "no-" ++ n else n
v = fromFlag t (def t)
```

Careful reader would notice that `showHide`

, `switchA`

and `switchB`

are *string-ly typed*. But that again, *depends*; not bad in this case. In the real codebase `showHide`

is used like:

```
<$> showHide ShowBuiltin "builtin" "Show / hide packages in global (non-nix-style) package db"
<*> showHide ShowGlobal "global" "Show / hide packages in nix-store"
```

Wrapping in additional `newtype`

won't help; adding `IsString`

instance would make newtype not any better than plain `String`

. On the other hand, if we use `DerivingVia`

to derive parsers, whole problem would go away.

To conclude, phantom types are quite useful, and if you spice them with a little of type-level programming, you won't need to write boilterplate and errorprone code.

]]>I'm happy to announce a new package: topograph. I says in its description:

Directed acyclic graphs can be sorted topographically. Existence of topographic ordering allows writing many graph algorithms efficiently. And many graphs, e.g. most dependency graphs are acyclic!

There are some algorithms build-in: dfs, transpose, transitive closure, transitive reduction... Some algorithms even become not-so-hard to implement, like a longest path!

Directed acyclic graphs are, as the name says, graphs without cycles. Such graphs are common in dependency management. In fact, my first use case was to analyse Haskell package dependencies.

Because there are no cycles, many graph-problems are much easier to solve than their general versions. A property is that graph can be topologically ordered; i.e. there is a linear ordering of its vertices, such that... (wikipedia). Or more concretely, we can assign each vertex an index, such that if `i > j`

, there **aren't** path from `i`

to `j`

.

The running example graph will be

It's a DAG. We can order vertices `ABXDE`

(or `AXBDE`

, check that both are valid topological orderings!). I marked one node `X`

to make it clearer, that you don't need to label nodes in the topological ordering :)

- Is there a path from
`E`

to`D`

?**No**.`4 > 3`

. - Is there a path from
`B`

to`X`

? If we use`ABXDE`

as the ordering, we know that**maybe**, we need to look closer.

`topograph`

have few algorithms built-in, e.g. transitive closure (added purple edge) and transitive reduction (green edges removed). Curiously, they are implemented using the same function parameterised by different predicate. Take a look at the source.

The library uses `runST`

-method to hide the fact that indices are actually `Int`

s, by requiring that algorithms are written over universal `i`

. For example the type of transitive closure is

At the end, you can run algorithms with the `runG`

function:

```
runG
:: forall v r. Ord v
=> Map v (Set v)
-- ^ Adjacency Map
-> (forall i. Ord i => G v i -> r)
-- ^ function on linear indices
-> Either [v] r
-- ^ Return the result or a cycle in the graph.
```

You pass in an adjacency map representation, an algorithm and you get either a `Right`

result, or a `Left`

if there were a cycle.

In a discussion on Twitter, Andrey Mokhov mentioned that there is a Summer of Haskell idea to add type-safe representation for acyclic graphs to `alga`

. When it gets implemented, we can add a variant of `runG`

which always succeeds!

Finally, I want to show a small, but real example where I used `topograph`

. That hopefully illustrates how relatively easy is to do stuff with `topograph`

.

```
stuff adjMap = either (fail . show) id $ runG adjMap $ \g -> do
-- take a closure and bring all the fields of G in scope
let G {..} = closure g
-- double loop
for_ gVertices $ \a -> for_ gVertices $ \b -> when (a < b) $ do
-- edge destinations
let ae = Set.fromList $ gEdges a
let be = Set.fromList $ gEdges b
-- an exercise: what is cs?
let cs | a `elem` be = [a]
| b `elem` ae = [b]
-- Here we are clever,
-- Set.minView and no foldr would be ok for some graphs
-- Why?
| otherwise = case Set.maxView $ Set.intersection ae be of
Nothing -> []
Just (x, xs) -> foldr f [x] xs where
f y zs = y : filter (`notElem` gEdges y) zs
-- print results
-- in real version this part is fancier
print (gFromVertex a, gFromVertex b, map gFromVertex cs)
```

An exercise is to find out what does `stuff`

do. Hint: I give a result of running it with an example graph above. The extra points are for explaining what that `Set.maxView`

and `foldr`

business is about.

A short gist about multi-argument things. Today is about *Prisms*.

```haskell
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE FunctionalDependencies #-}
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE TypeFamilies #-}
module ReviewN where
import Control.Lens
import Data.Tuple.Only
```

Let's consider an example type and its autogenerated `Prism`

s.

The `_Foo`

is a nice `Prism' FooBarBaz Int`

. For example, we can use it to construct `FooBarBaz`

But `_Bar`

is unpleasant `Prism' FooBarBaz (Int, Char)`

. We have to tuple

Can we do better?

We can write helpers

```
review2 :: AReview t (b1, b2) -> b1 -> b2 -> t
review2 l b1 b2 = review l (b1, b2)
review3 :: AReview t (b1, b2, b3) -> b1 -> b2 -> b3 -> t
review3 l b1 b2 b3 = review l (b1, b2, b3)
```

but those aren't satisfying.

Luckily, `Prism`

carries "amount of arguments", so we can write a a quite simple type class to abstract over `review2`

, `review3`

...

```
class ReviewN tuple where
type ReviewNArgs tuple t :: *
reviewN :: AReview t tuple -> ReviewNArgs tuple t
instance ReviewN (b1, b2) where
type ReviewNArgs (b1, b2) t = b1 -> b2 -> t
reviewN = review2
instance ReviewN (b1, b2, b3) where
type ReviewNArgs (b1, b2, b3) t = b1 -> b2 -> b3 -> t
reviewN = review3
```

So now we can write nicer auto-uncurried code:

That's an improvement

Note that if we try to define an infix operator:

it won't work as we want. We'd need a fixity tighter than juxtaposition. :(

Let's not forget the nullary constructor `Baz`

. It generates `_Baz :: Prism' FooBarBaz ()`

.

Our `ReviewN`

can easily handle this too: `()`

is an empty tuple:

But how about the very nice `_Foo`

. There are `No instance for (ReviewN Int) arising from a use of ‘reviewN’`

. Do we have to switch between `review`

and `reviewN`

?

One solution is to use `Only`

(or `Identity`

or ...)

```
instance ReviewN (Only b) where
type ReviewNArgs (Only b) t = b -> t
reviewN l = review l. Only
_Foo' :: Prism' FooBarBaz (Only Int)
_Foo' = _Foo . coerced
```

Using `_Foo'`

prism, `reviewN`

works fine too:

This is a very small example of somewhat generic programming. I think this (obvious?) trick might be useful in DSLs where "functions" of the DSL can have multiple inputs (or outputs!) - there we can choose to wrap single parameters in `Only`

, which would allow to abstracti over arity.

`Prism`

(and optics) happen to be quite general DSL ;)

Also I'm pretty sure that this can be made work with `generic-lens`

machinery, so in a single argument case `(Only b)`

is generated, so we could write:

Which is silly as we can just

but again, you never know where that could be useful.

]]>In this post I’ll try to convince you that optic laws (as in the `lens`

package documentation) are the right ones. For example recall the `Lens`

laws (law names come from alternative naming of the operations: `view = get`

and `set = put`

^{1}.

**GetPut**: You get back what you put in:**PutGet**: Putting back what you got doesn’t change anything:**PutPut**: Setting twice is the same as setting once:

Why there are these three laws (why three?). How we know these three laws are "enough", or that they don’t require too much?

The solution is to consider a representation which is obviously a `Lens`

(or `Prism`

or ...), but for which also the "laws" are obvious. Then when we write conversion functions between this and more concrete representation we’ll find out what laws are needed.

The `Iso`

- an isomorphism is the simplest example. The concrete representation is

But the an obvious definition in math world (citing Awodey 2010): In any category , an arrow is called an *isomorphism*, if there is an arrow in such that and . We write if there exists an isomorphism between and .

Here, the mathematical conditions tells precisely which laws are required, namely for a (`RecordWildCards`

destructed) `Iso`

fields:

The isomorphism case is trivial. In the post I’ll go through more interesting optics:

`Lens`

and`Prism`

. Their laws should not be surprising to people familiar with`lens`

library. However we discover a new law for`Prism`

.`Affine`

(traversal). I discussed affine traversal in a post about them and give laws in Glassery. These laws were written based purely on intuition, which we’ll now verify.`Achroma`

tic lens. There are my quickly written notes from March 2018. Turns out, I missed few bits.`Grate`

. Russel O’Connor gives laws for a van Laarhoven Grate (a two functor variant) in Grate: A new kind of Optic. We formulate them for "normal" Grates:`grate :: ((s -> a) -> b) -> t`

. In the process we also learn more about`Grate`

.

At the end we discuss a little about `Setter`

s. And conclude the post with general thoughts.

The Isomorphism lenses by Twan van Laarhoven makes the same things for lenses; we do it for more classes of optics.

A *Lens* is a purely functional reference. Concrete `Lens`

representation is

**Note:** Here and later we work with so called monomorphic optics, i.e. optics which don’t allow type change. Hopefully I will lift this restriction in a future blog post. Also we swapped the arguments for `set`

(in `lens`

library `set :: ASetter s t a b -> b -> s -> t`

). This version is easier to do math with (we will partially apply it).

We claim that a `Lens`

*should be* a subject to the following three laws:

`view (set s a) ≡ a`

`set s (view s) ≡ s`

`set (set s a) a' ≡ set s a'`

Let’s show that the laws are both *sound* (not asking too much) and *complete* (asking enough). For that we’ll need another representation of lenses: an existential formulation:

This formulation isn’t novel (e.g. Russel O’Connor have a footnote about affine traversal in *A representation theorem for second-order functionals*), but I haven’t seen it used to highlight the laws. In Lenses Are Exactly the Coalgebras for the Store Comonad the laws of a coalgebra for a comonad are used. As my category theory knowledge is not very deep, I find the "coalgebra for a comonad" intimidating. I don’t know if and how "Lenses are the coalgebras for the costate comonad" is generalisable to other optics. Existential formulation on the other hand works *for everything*.

We can define `view`

and `set`

operations using existential formulation:

where is the forward direction, and is the backward direction of the isomorphism.

We can prove that `Lens`

laws are *sound* by showing that this definitions satisfy the laws. For example the first **GetPut** law:

The remaining **PutGet** and **PutPut** laws can be proved sound using similar direct calculation. In fact, I mechanized the proofs in this post; and soundness properties are proven mostly by a very simple tactic.

The other direction, *completeness*, is more interesting. Given and , such that the three laws above hold, construct a .

The key insight is to pick the right "left overs" type . I found the process of mechanizing proofs helpful, you will immediately rule out e.g. bare . The (partially applied ) is better, but still too "big". We need to use a smaller, *refined* type:

All three options make sense if you consider how `lens :: (s -> a) -> (s -> b -> t) -> Lens s t a b`

can be defined using e.g. `Profunctor`

encoding, Glassery uses:

```
lens :: (s -> a) -> (s -> b -> t) -> Lens s t a b
lens getter setter pab = dimap
-- c = s
(\s -> (getter s, s))
(\(b, s) -> setter s b)
(first' pab)
```

but you can also write

```
lens :: (s -> a) -> (s -> b -> t) -> Lens s t a b
lens getter setter pab = dimap
-- c = b -> t; or refined
-- c = { f : b -> t | exists s. f = setter s }
(\s -> (getter s, setter s))
(\(b, f) -> f b)
(first' pab)
```

Anyway, after a is found, we can define an isomorphism. The arrows are defined as in the latter `Profunctor`

example:

The proof obligation for the refinement is not really an obligation at all, the equality is definitionally true (). However, next we need to prove the isomorphism properties. The backwards direction is easy, it’s essentially a **PutGet**-law:

Forward direction is more involved. Here we will use proof irrelevance for the refinement proofs. Those proofs aren’t computationally relevant, so this is justified. I think it’s possible to encode these proofs in Cubical Agda. I also use functional extensionality, but it’s a theorem in Cubical Agda (it’s used once in the following part of the proof, try to spot it!) We will need to use both **PutPut** and **GetPut** laws, for the both halves of the product (pair), respectively.

We continue by proving the equality of pair halves separately:

and the other half:

This completes the completeness proof. We had to use all three laws. I’m sure that intuitively thinking the **PutPut**-law is simple to forget, but these calculations show that it is important.

In previous subsections, we knew the right `Lens`

laws (candidates) a priori. We only needed to verify that they are in fact the right ones. But what if we didn’t knew the laws beforehand. Could we somehow calculate them?

In the completeness proof, all branches ended up with the use of some law, however we *knew* which to pick, so we end up with a proof goal which is exactly one of the laws we can use. Therefore, trying to write a completeness proof is not a direct calculation way, either we need to know the right existential type to use (in my opinion not entirely obvious), or which laws there possibly are.

Unfortunately I don’t have a definitive answer to this problem. A sketch of the idea is however to start with

isomorphism, and try to massage it into something without . Notice that we work in a category where all arrows are isomorphisms (groupoid), therefore we use -tensor, not product.

If we assume that we have a Symmetric closed monoidal groupoid, then we can use a property:

to find out that and are isomorphic. (And this is the hand-wavy part: I don’t understand groupoids enough to say if above reasoning make any sense at all). We need to "remember" how to go back in direction, not all are invertible (e.g. `const someS`

). That’s one reasoning to refinement of .

We could stop here, as we in fact calculated what existential we have to use (or at least could try). We can also continue, at this point we have

-less representation of `Lens`

. If we *forget* the isomorphisms, the remainders are and arrows (when we split the product), these are and arrows! So we went from existential to concrete representation of `Lens`

without using Yoneda. (What you needa know about Yoneda is an interesting reed anyway!)

But if we stick with this representation, then we can write down the laws too: If left-to-right direction is and the right-to-left is (here I hand-wave about, what the first half of the pair should be!) then we get

Note how the **PutPut**-law is in contracted form, the form which we would preferred in the above `Lens`

laws completeness proof (spoiler: that’s where I used functional extensionality, if you haven’t spotted it).

In this section we do similar soundness and completeness proofs for prisms. A *Prism* is a generalized constructor. Concrete `Prism`

representation is (`preview`

and `review`

are often called `match`

and `build`

respectively, I stick with `lens`

-library vocabulary).

We claim that a `Prism`

*should be* a subject to the following two laws:

**MatchBuild**: First, if I`review`

a value and then`preview`

I will get it back:`preview (review a) ≡ Just a`

**BuildMatch**: Second, if you can extract a value`a`

using a`preview`

from a value`s`

, then the value`s`

is completely described by a prism and`a`

:`preview s ≡ Just a ⇒ review a ≡ s`

The existential formulation of a `Prism`

is similar to `Lens`

’, but instead of product we have a coproduct (or sum):

First step is to define `preview`

and `review`

operations for existential representation:

where is the forward direction, and is the backward direction of the isomorphism.

As with `Lens`

none of the laws is hard to show sound. The first one is a direct computation:

and for the second one we need to prepare the precondition. The injectivity of constructors let’s rule out the case when , because .

using which we can prove the law itself

We need to construct a from the given and . Here also, as in the `Lens`

case, we need to pick the right type for the existential. And again we need a refined type:

And yet again, that type can be found either from `Profunctor`

definition of `prism`

(the `either id setter`

):

```
prism :: (b -> t) -> (s -> Either t a) -> Prism s t a b
prism setter getter = dimap getter (either id setter) . right'}
```

or by informal calculation

Then we can define the arrows of the isomorphism (I’m quite sure that smart enough proof search would find exactly these definitions):

We use non-standard ^{2}, which provides the match proofs too. yet it can be directly coded e.g. in Agda as:

```
open import Data.Maybe
open import Relation.Binary.PropositionalEquality
maybe-case : ∀ {a b} {A : Set a} {B : Set b}
→ (m : Maybe A)
→ (m ≡ nothing → B)
→ ((a : A) → m ≡ just a → B)
→ B
maybe-case (just a) f g = g a refl
maybe-case nothing f g = f refl
```

Next we can show that and are inverses. The case:

which we can split:

and the second case

The other direction of the isomorphism, has two cases: first :

and

The `Prism`

law proofs are only slightly more complex than `Lens`

law; mainly due the need to do case analysis.

One interesting point, is that we might want to have third, **MatchMatch** law. There is polymorphic `matching :: APrism s t a b -> s -> Either t a`

combinator. Using its (non-existing in `lens`

) monomorphic version `matching' :: APrism' :: s -> Either s a`

version we can write:

The law is silly, as it completely defines the left part of the `Either`

, therefore we can have `Maybe`

in `preview`

without any loss.

For example take a bogus `_Right`

`Prism`

variant:

```
_Right' :: Prism' (Either Int a) a
_Right' = prism Right $ \t -> case t of
Right x -> Right x
Left i -> Left (Left (i + 1))
```

which will pass the `QuickCheck`

tests:

```
matchBuild_prop :: Int -> Property
matchBuild_prop x = preview _Right' (review _Right' x) === Just x
buildMatch_prop :: T Int -> Property
buildMatch_prop s = case preview _Right' s of
Just a -> label "just" $ review _Right' a === s
Nothing -> label "nothing" True
```

but behave weirdly

or

However, the **MatchMatch** property is clearly violated:

```
matchMatch_prop :: T Int -> Property
matchMatch_prop s = case matching _Right' s of
Right _ -> label "right" True
Left t -> label "left" $ t === s
```

as `QuickCheck`

points out:

Our verification process didn’t needed this law, as we used `preview`

and `review`

as core `Prism`

primitives. There are two ways to construct `Prism`

in the `prism`

library:

```
prism :: (b -> t) -> (s -> Either t a) -> Prism s t a b
prism' :: (b -> s) -> (s -> Maybe a) -> Prism s s a b
```

we use the latter `prism'`

way. As you see it can construct only semi-monomorphic `Prism`

. I think it’s type should be restricted to return `Prism s s a a`

. And we need to check the **MatchMatch** law for `Prism`

s constructed with `prism`

. If we used (even the monomorphic version of) `matching`

in the completeness proof, we would see the need for the law, if we actually use the left component of the result.

The **MatchMatch** law as stated doesn’t generalize to the polymorphic variant, as we require `s ~ t`

, to be able to compare them.

My original idea of the law doesn’t work, neither another way to state it:

```
-- too weak
matching p s ≡ Left t ⇒ matching p t ≡ Left t'
matching p s `leftBind` matching p' ≡ matching p s
```

A counterexample is when all of negative matches are the same:

```
_Right'' :: Prism' (Either Int a) a
_Right'' = prism Right $ \t -> case t of
Right x -> Right x
Left i -> Left (Left 0)
```

However one polymorphic variant of **MatchMatch** law:

seems to catch all invalid examples I can come with. It’s not directly enough for *Coq* (maybe it is). The relaxation of strict equality to `matching p t ≡ Left s`

is sensible, as intuitively there should be a way to convert `t`

back to `s`

, `t`

(as well as positive match results) fully determine the original `s`

.

I discussed affine traversal previously in a post about them and gave laws in Glassery.

You get back what you put in:

`preview l (set l v s) ≡ v <$ preview l s`

If you can extract a value, and put it back, that doesn’t change anything.

`preview l s ≡ Just a ⇒ set l s a ≡ s`

If you get nothing when extracting a value, then whatever you put in, the operation is no-op.

`preview l s ≡ Nothing ⇒ set l s a ≡ s`

Setting twice is the same as setting once:

`set l v' (set l v s) ≡ set l v' s`

Turns out that the third law is not required in monomorphic setting, similarly as **MatchMatch**-laws isn’t needed in the `Prism`

case.

In this section I won’t show the full proofs, you can check (step through) the *Coq* sources at https://github.com/phadej/lens-laws. Let’s however highlight the main points.

The concrete representation is the "worst" of `Lens`

and `Prism`

(where `Iso`

would be the best of both):

The existential representation explains the naming of this optic:

Defining operations using existential representation, and proving soundness of the laws is a combination of above `Lens`

and `Prism`

proofs.

The completeness proof is also similar to `Lens`

and `Prism`

proofs. The interesting part is what’s the value of existential types, given and we use:

These types are (again) very similar to the ones used in `Prism`

and `Lens`

completeness proofs.

At this point it’s good time to discuss some properties of optics we have covered so far:

any

`Iso`

can be turned into`Lens`

,`Prism`

or`Affine`

any

`Lens`

can be turned into`Affine`

any

`Prism`

can be turned into`Affine`

each class of optics is closed under composition

First three properties should be easy to see. For example take a , we can turn it into by taking (`Void`

in Haskell).

The last property is not hard to verify either. is clearly closed under composition (transitivity), and need associativity of and , i.e.

The `Affine`

case is slightly trickier:

Imagine (or try to write or look into *Coq* proofs) the needed plumbing in the existential (or concrete) lens representations! It’s very neat, that van Laarhoven and `Profunctor`

encodings use uniform representations so all optics can share the same composition combinator, and that encoding is based on type-classes allowing automatic "upcasting" of optics. The downside is that encoding internals leak into error messages (though https://github.com/mrkgnao/silica demonstrates how that can be improved!)

Russell O’Connor and James Deikun discovered an optic called `Grate`

^{3} in Grate: A new kind of Optic. Using `Grate`

we can `zipWithOf`

structures. Essentially `Grate s a`

witnesses that `s`

is some kind of `Distributive`

or `Representable`

containing (always the same amount of) `a`

s, and nothing else.

It sounds like `Traversal`

, but this optic is very different. The concrete representation is obscure:

However its existential representation is very simple (I guess the `Closed`

subclass of `Profunctor`

helped the discovery to happen):

Our tasks is to discover the laws. Russell doesn’t formulate them for concrete representation, checking `Distributive`

documentation doesn’t help either. We can write grate using existential encoding:

where is the forward direction, is the backward direction of the isomorphism. Having this around will help us to rule out too strong laws, the one we cannot fulfill (i.e. violate soundness).

The journey starts with a short detour. It turns out that stating the laws for `tabulate`

and `index`

construction is a lot easier. Also in this case we stay in *Haskell98* as we don’t need type families (`Representable`

type-class has associated `type Rep f`

) the index type will be existentially hidden away, i.e:

```
fromRepresentable :: (s -> i -> a) -> ((i -> a) -> s) -> Grate s a
fromRepresentable index tabulate = _exercise
```

is not hard to implement, and the required properties of `index`

and `tabulate`

are almost obvious: they have to be inverses.

If you stare at `Grate`

concrete representation and the types of `index`

and `tabulate`

(and implementation of `fromRepresentable`

), you might see that `s -> a`

acts as `i`

, i.e. `grate`

type is the similar to `tabulate`

’s "give me a function from indices to elements, and I’ll return you a whole container". This gives us something to try for a completeness proof:

We can define (possible) arrows of the isomorphism using , the implementation is (suspiciously) simple:

We have try to write a proof to see what properties must have to make and actually inverses.

This proof obligation seems reasonable to require as the law, it’s a special case of `tabulate (\i -> index a i)`

`Representable`

law:

**TabulateIndex**: Tabulating by indexing doesn’t change anything.

So far everything looks good. Let’s try the other direction:

Can we require that as a law?

**IndexTabulate**: Indexing into tabulated value gives a value at the index?

The answer is yes and no. There is a but. If we require the law as such, it’s too strong. `i`

and `f`

are arbitrary!

```
import Data.Functor.Identity
import Test.QuickCheck
import Test.QuickCheck.Instances
newtype Grate s a = Grate { grate :: ((s -> a) -> a) -> s }
identityGrate :: Grate (Identity a) a
identityGrate = Grate (\f -> Identity (f runIdentity))
law2example :: (Fun (Identity Int) (Int)) -> Property
law2example (Fn i) = i (grate identityGrate f) === f i where
f :: (Identity Int -> Int) -> Int
f = const 42
```

I claim that `identityGrate`

is lawful `Grate`

, however `lawExample`

`Property`

doesn’t hold, `QuickCheck`

finds a counterexample immediately. `f`

indeed can be arbitrary, but the indices `i`

aren’t! Only some values of type are used, in `identityGrate`

a single one: `runIdentity`

.

We have make small amendments to the previous. Each grate will have a associated predicate and the correct existential type is

Definitions of needs to be slightly changed to forget the refinement:

The need to be adjusted to, or actually the itself, its more precise type is (remember that in "good" languages, like LiquidHaskell, refinement types are erased, i.e. don’t have any runtime cost!)

The user of the library "should" provide a predicate and verify that passes only functions satisfying that predicate. The final versions of two laws are:

**TabulateIndex**: Tabulating by indexing doesn’t change anything.**IndexTabulate**: Indexing into tabulated value gives a value at the index?

For example, for above `Identity`

example such predicate can be simply (can we encode this in LiquidHaskell, I remember seeing some predicate support?)

```
-- this holds (almost trivially)
law2proper :: Int -> Property
law2proper x = runIdentity (grate identityGrate f) === f runIdentity where
f :: (Identity Int -> Int) -> Int
f = const x
```

In general, for finite containers enumerating the all possible index values is the simplest way to define predicate . There are *some* indices, though the implementation hides them.

Careful reader should notice that we have to do soundness proofs still. We have two parts of the isomorphism is the forward direction, is the backward direction for some existential index type ; is defined using them.

**TabulateIndex** holds almost trivially:

For **IndexTabulate** we have to define what is the for the existential formulation. We check how is implemented (the unfolding is just in the above proof), what’s the argument of is called with:

Using this predicate, we can verify the law:

This completeness the soundness proof. We now have discovered the `Grate`

laws. As an exercise you could think about

```
data V2 a = V2 a a
_V2 :: Grate (V2 a) a
_V2 = _grateExercise1
silly :: Grate () a
silly = _grateExercise2
```

Achromatic lenses are "A lost of the bidirectional programming folks like a variant of lenses that have get, put, and create." (quote by Russell O’Connor). Originally, in my notes on achromatic lenses the concrete representation has three operations, I rename them all to be nicely six characters long and reflect the database-holy-grailness: (update could have `s -> (a -> a) -> s`

type as well).

I listed four laws, three `Lens`

laws and one from `Prism`

or `Iso`

However I didn’t notice to specify the last, fifth law.

**SelectUpdate****UpdateSelect****UpdateUpdate****SelectInsert****UpdateInsert***new*

The existential representation for achromatic lens is almost like lenses’:

It can be understood that might have or none of it at all. It turned out that to create from three lawful components, we also needed an extra bit:

in other words, we *need to know*, for each , whether it has those extra bits .

In the completeness proof I used the following value for :

In Haskell implementation we don’t care:

```
type Optic p s t a b = p a b -> p s t
class Profunctor p => PointedCartesian p where
pointedSecond :: p a b -> p (Maybe c, a) (Maybe c, b)
achroma
:: forall p s t a b. PointedCartesian p
=> (s -> a) -- ^ select
-> (b -> t) -- ^ insert
-> (s -> b -> t) -- ^ update
-> Optic p s t a b
achroma sa bt sabt
= dimap f g
. pointedSecond
where
-- note: always Just.
f :: s -> (Maybe (b -> t), a)
f s = (Just (sabt s), sa s)
g :: (Maybe (b -> t), b) -> t
g (Nothing, b) = bt b
g (Just bt', b) = bt' b
```

We could try to implement a combinator checking whether the value is "pristine", i.e. could been created with `insert`

directly:

```
newtype Pristine a b = Pristine { unPristine :: a -> Bool }
instance Profunctor Pristine where
dimap f _ (Pristine h) = Pristine (h . f)
instance PointedCartesian Pristine where
pointedSecond (Pristine h) = Pristine $ \(m, a) -> isNothing m && h a
pristine :: Optic Pristine s s a a -> s -> Bool
pristine o s = unPristine (o (Pristine (const True))) s
```

But with our current version of `achroma`

, `pristine`

always returns `False`

. If we add `pristine`

to `achroma`

:

```
-- | better? concrete representation
data Achroma' s a = Achroma'
{ select' :: s -> a
, update' :: s -> a -> s
, insert' :: a -> s
, pristine' :: s -> Bool
}
-- | better? Profunctor encoding constructor
achroma'
:: forall p s t a b. PointedCartesian p
=> (s -> a) -- ^ select
-> (b -> t) -- ^ insert
-> (s -> b -> t) -- ^ update
-> (s -> Bool) -- ^ pristine
-> Optic p s t a b
achroma' sa bt sabt pris
= dimap f g
. pointedSecond
where
f :: s -> (Maybe (b -> t), a)
f s = (if pris s then Nothing else Just (sabt s), sa s)
g :: (Maybe (b -> t), b) -> t
g (Nothing, b) = bt b
g (Just bt', b) = bt' b
```

then `pristine`

combinator will work, whenever way the value is created or modified. The `f`

and `g`

above are essentially the same as used in the completeness proof.

Addition of `pristine`

will add two more laws (or one, depends on how you count), which are encoded in the type of :

**Pristine**or alternatively as a variant of

`Prism`

’s**BuildMatch**law^{4}**NotPristine**

The achromatic lens case illustrates again why it’s important to think about laws; when making notes I didn’t think about `pristine`

case, for example.

The last section is about `Setter`

s. They have only single operation `over`

:

What is its existential representation? Look at all other different optics we examined:

Hopefully the pattern stands out, namely is isomorphic to for some *endofunctor* :

Endofunctors can be composed, and there’s an identity functor. All optics seems to be given by some (closed under composition) class of endofunctors. For example, `Traversal`

is given by `Traversable`

functors (whatever these are in category theoretical terms).

That’s the point which makes `Profunctor`

encoding quite elegant, Look at the (stripped of) classes behind the implementation (there are no class for `Affine`

, it’s made simply by a `(Strong, Choice)`

```
class Profunctor p where dimap :: (a -> s) -> (t -> b) -> p a b -> p s t
class Strong p where second' :: p a b -> p (c, a) (c, b)
class Choice p where right' :: p a b -> p (Either d a) (Either d b)
class Closed p where closed :: p a b -> p (i -> a) (i -> b)
class PointedC p where psecond' :: p a b -> p (Maybe c, a) (Maybe c, b)
-- Setter
class Mapping p where map' :: Functor f => p a b -> p (f a) (f b)
-- Traversal
class Traversing p where traversed :: Traversable f => p a b -> p (f a) (f b)
```

It also explains why various `Functor`

classes are enough to implement *simple* (i.e. not type-changing, as in this post) optics, described in Functor optics -post.

Now is possible to consistently invent (or discover?) new optics, for example `LensGrate`

, combination of `Lens`

and `Grate`

(I remember seeing it in some library):

Does it have some handy operations still, or does it degenerate to `Setter`

? Or `PrismGrate`

, which can still `review`

things: try to verify that this class is closed under composition:

I think `AffineGrate`

is practically as useful as `Setter`

, but you can still speak about it. `AchromaticGrate`

seems to be useful concept, we can implement a variant of `view`

for it, as there’s a single index which stands out (and e.g `silly`

isn’t an `AchromaticGrate`

):

Or we can have `FiniteGrate`

, where index is some finite set. Representing the index with gives us an *order*, so we can `traverseOf`

`FiniteGrate`

. However, `Traversal`

’s aren’t `FiniteGrate`

s (exercise!)

So many new or at least not well known optics, that we completely forgot to talk about `Setter`

laws. They aren’t surprising, when we know the existential encoding:

**OverIdentity****OverComposition**

Proofs of their soundness and completeness can be found in Coq formalisation, or you can try to write them yourself as an exercise.

In this blog post we presented a results of formalization of some optic laws. Thinking about laws paid off, among other things we

discovered

**MatchMatch**-law for`Prism`

s,discovered a

**UpdateInsert**law, as well as possible new core operation`pristine`

for`Achroma`

s.formulated laws for

`grate`

operation of`Grate`

.

Also in the `Setter`

section we discussed some other not well known optics.

A lot from a single insight of requiring the isomorphism (and not only a equivalence, i.e. bare back and forth arrows without requiring them to be inverses) in the existential formulation of optics.

Hopefully you enjoyed the read, and this blog post gave you some food for thought!

but

`get`

and`put`

clash with`MonadState`

names↩Writing the definition using (dependent)

`Maybe`

elimination`maybe-elim : ∀ {a b} {A : Set a} → (P : Maybe A → Set b) → P nothing → ((x : A) → P (just x)) → (m : Maybe A) → P m maybe-elim P n j (just x) = j x maybe-elim P n j nothing = n`

is a nice exercise. Something like that is what

`remember`

tactic would do in*Coq*. Knowing what happens helps to write definitions by hand.↩Russell writes "Until Edward Kmett comes up with a better name, let’s call this new optic a

`Grate`

", three years later we are probably stuck with that name.↩in my previous notes I though that law doesn’t hold: I didn’t notice to specify the precondition↩

This simple idea hit me when I was watching a talk about (dynamic) linking. On Linux (at least) we can `LD_PRELOAD`

**own** shared objects, to override symbols in other (maybe even proprietrary) shared objects. This is useful for various purposes. Can we do something like that in Haskell?

Not exactly, Haskell is not C. But, as the ecosystem is open, we can simply vendor in the dependency (as source) and modify it. E.g. add some debug prints to figure out why something doesn't work as expected? We *preload* on the source level, i.e. use the local version of package, not the one on the Hackage.

However we don't need to stop there. We can mess with the vendored packages as much as we need to. Imagine that we want to use `github`

package, but with `HsOpenSSL`

for encryption, not the `tls`

package.

There is a proof-of-concept implementation on GitHub phadej/preload-trick. The example program needs small adjustement, because every application that uses `HsOpenSSL`

must wrap any operations involving OpenSSL with `withOpenSSL`

:

```
{-# LANGUAGE CPP #-}
{-# LANGUAGE OverloadedStrings #-}
module Main (main) where
import qualified GitHub as GH
#ifdef MIN_VERSION_HsOpenSSL
import OpenSSL (withOpenSSL)
#else
withOpenSSL :: IO a -> IO a
withOpenSSL = id
#endif
main :: IO ()
main = withOpenSSL $ do
possibleUser <- GH.executeRequest' $ GH.userInfoForR "phadej"
print possibleUser
```

This example works out of the box with `http-client-tls`

. It turns out that we need to *mock* only three packages, if we want to use `HsOpenSSL`

:

**http-client-tls:**This is obvious.`http-client-tls`

relies on`tls`

for TLS implementation. Luckily`github`

uses only`tlsManagerSettings`

which can be implemented as oneliner using`http-client-openssl`

!`module Network.HTTP.Client.TLS (tlsManagerSettings) where import Network.HTTP.Client (ManagerSettings) import Network.HTTP.Client.OpenSSL (opensslManagerSettings) import qualified OpenSSL.Session as OpenSSL tlsManagerSettings :: ManagerSettings tlsManagerSettings = opensslManagerSettings OpenSSL.context`

**tls:**It turns out`github`

doesn't use anything from`tls`

directly. The dependency definition exists there to forbid old versions of`tls`

(`git blame`

points to commit that originally added`tls >=1.3.5`

constrait. That version of`tls`

has*Fix a bug with ECDHE based cipher where serialization*, I don't remember whether that is important or not, better to be safe). As`tls`

isn't used, the mock package is an empty package (I specified a lot more than needed):