LChart: displaying charts in F# – Part III

The last post is here. In this post we’ll look at how things work under the cover and to why I came to believe that they shouldn’t work this way.

First of all each one of the functions to create charts looks something like this:

static member bar (?y,?x, ?isValueShownAsLabel, ?markerSize, ?markerStyle, ?color, ?xname, ?yname, ?seriesName, ?title, ?drawingStyle) = 
    let c = Create (SeriesChartType.Bar, x, y, isValueShownAsLabel, markerSize, markerStyle, color, xname, yname, seriesName, title)
    c.Series.[0].["DrawingStyle"] <- defaultArg drawingStyle (c.Series.[0].["DrawingStyle"])
    c

This returns an object of type lc (this is the type of ‘c’). But lc inherits from Chart which is the main class in the Microsoft Chart Controls.

type lc() =
    inherit Chart()

I should have said at the start that you need to reference such controls.

#r "System.Windows.Forms.DataVisualization.dll"
open System.Collections
open System.Drawing
open System.IO
open System.Windows.Forms
open System.Windows.Forms.DataVisualization.Charting
open System.Windows.Forms.DataVisualization.Charting.Utilities
open System

It is convenient that the return value of each function is a subtype of Chart. You can then go and customize this object as you like (i.e. changing graphical appearance) before calling display. Given the Chart inherits from Control you can code the display method as follows:

let display (c:lc) =
    let copy () =
        let stream = new MemoryStream()
        c.SaveImage(stream, Imaging.ImageFormat.Bmp)
        let bmp = new Bitmap(stream)
        Clipboard.SetDataObject(bmp)
    
    c.KeyDown.Add(fun e -> if e.Control = true && e.KeyCode = Keys.C then copy ())
    let pressToCopy = "(press CTRL+C to copy)"     
    let name = if c.Titles.Count = 0 then sprintf "%s %s " "lc" pressToCopy else sprintf "%s %s " c.Titles.[0].Text  pressToCopy
    let f = new Form(Text = name, Size = new Size(800,600), TopMost = true)
    c.Dock <- DockStyle.Fill
    f.Controls.Add(c)
    f.Show()
    c

Apart from a bit of convolutions to implement a Copy function, this just put the Chart control on a newly created Form. The Create method called inside bar looks like the following.

static let Create (chartType, x, y, isValueShownAsLabel, markerSize, markerStyle, color, xname, yname, seriesName, title) =
    let c = new lc()
    let a = new ChartArea()
    let s = new Series()
    s.ChartType <- chartType
    c.ChartAreas.Add(a)
    c.Series.Add(s)
    match x, y with
        | Some(x), None     -> failwith "You cannot pass only x to a chart drawing function"
        | Some(x), Some(y)  -> s.Points.DataBindXY(x, [|y|])
        | None, Some(y)     -> s.Points.DataBindY([|y|])
        | None, None        -> ()
    s.IsValueShownAsLabel <- defaultArg isValueShownAsLabel s.IsValueShownAsLabel
    s.MarkerSize <- defaultArg markerSize s.MarkerSize
    s.MarkerStyle <- defaultArg markerStyle s.MarkerStyle
    s.Color <- defaultArg color s.Color
    
    a.AxisX.MajorGrid.Enabled <- false
    a.AxisY.MajorGrid.Enabled <- false
    
    match xname with
    | Some(xname) ->
        a.AxisX.Title <- xname
        a.AxisX.TitleFont <- axisFont
        a.AxisX.TitleForeColor <- axisColor
    | _ -> ()
    match yname with
    | Some(yname) ->
        a.AxisY.Title <- yname
        a.AxisY.TitleFont <- axisFont
        a.AxisY.TitleForeColor <- axisColor
    | _ -> ()
    match seriesName with
    | Some(seriesName) -> s.Name <- seriesName
    | _ -> ()
    match title with
    | Some(title) ->
        let t = c.Titles.Add(title: string)
        t.Font <- titleFont
        t.ForeColor <- titleColor
    | _ -> ()
    c

Pretty standard imperative code here. Creating a chart and assigning its properties. Read the documentation for the Chart Control to understand what I’m doing here. I’m not even sure I remember what I’m doing. Given that we have our own lc class (which is a type of Chart) we can then override the ‘+’ operator and ‘++’ operator to do what is needed.

static member (+) (c1:lc, c2:lc) =    
    let c = copyChart(c1)   
    c1.ChartAreas |> Seq.iter (fun a -> addAreaAndSeries c a c1.Series)
    let lastArea = c.ChartAreas |> Seq.nth ((c.ChartAreas |> Seq.length) - 1)
    c2.Series |> Seq.iter(fun s -> c.Series.Add(copySeries s c lastArea.Name))
    let l = c.Legends.Add("")
    l.Font <- legendFont
    c
   
static member (++) (c1:lc, c2:lc) =
    let c = copyChart(c1)   
    c1.ChartAreas |> Seq.iter (fun a -> addAreaAndSeries c a c1.Series)
    let lastArea = c.ChartAreas |> Seq.nth ((c.ChartAreas |> Seq.length) - 1)
    addAreaAndSeries c c2.ChartAreas.[0] c2.Series
    let firstArea = c.ChartAreas |> Seq.nth ((c.ChartAreas |> Seq.length) - 1)
    c2.ChartAreas |> Seq.skip 1 |> Seq.iter (fun a -> addAreaAndSeries c a c2.Series)
    c    

Apart from some other utility functions, this is how it all works. Why do I say that it is wrong? It is my opinion that the right way to do it would be to use ‘+’, ‘++’ and all the lc.XXX functions to create an object model that is completely independent from the Microsoft Chart controls. The display method would then translate it to the appropriate displayable Chart. It would work like a compiler translating to IL and then a Jitter producing native code. This would:

  • Make possible to do more interesting compositions of graphs. Now I’m very constrained in what I can do by the fact that I’m working directly with Chart objects
  • Make possible to change the backend. Using something different than Microsoft Chart controls to draw the chart

Why I have not done it? I didn’t know that was the right design until I used the wrong one. Now that I know, I have no time to do it.

Luca Bolognese leaves Microsoft

FYI I imported all the posts from the old MSDN blog to this one. Hence this post doesn’t make sense anymore. I left it here for memory sake. BTW: I am also back in msft now (30/11/2018).

This is my last post on this blog. My new blog is here: https://lucabolognese.wordpress.com/

I accepted a role as Director for Credit Suisse in London. I’m excited by the opportunity to work in the financial industry, a long-standing desire of mine. I’m also excited to write more F# code and to be closer to Italy, where my extended family is.

The past ten years in Microsoft have been a wild ride. I’m proud to have been part of ObjectSpaces, Generics, LINQ and F# (and much more …). I’ve been lucky to be able to post on this blog and present at conferences about such innovative technologies. It has all been a lot of fun. I’m sure the next ten years will be as good.

I’ll see you guys on my new blog.

LChart: displaying charts in F# – Part II

In the previous post on my old blog I showed how to display simple charts with LChart. In this one we’ll talk about more complex charts. I wanted to define a little language for graphs for the sake of creating a more complex chart in a single line of code. Remember, the scenario here is: I got some data, I want to display it quickly in the fsi. The language has two operators: ‘+’ and ‘++’.

‘+’ allows you to to superimpose things on a chart as in the following example.

lc.scatter(y) + lc.line(y) |> display

Notice how I can superimpose two graphs of a different type and display the result.

image

Notice on the upper right corner a bizarre (Series1, Series2) legend. What I wanted to do, but didn’t get around to do, was to allow a syntax like

lc.scatter(y) + lc.line(y) + lc.legend(“Players”) |> display

It should be relatively trivial to add it to the code. Also notice that the ‘y’ parameter in the second chart is not needed. Data flows freely from left to right. So you can write the equivalent code below. This is a feature that caused all sort of grief. With hindsight it’s not worth the hassle. That’s why you always need to write your code at least twice to get it right.

lc.scatter(y) + lc.line() |> display 

Some other, more elaborate charts follows. Notice how data flows from left to right until I introduce a new variable (i.e. the two lc.boxplot  instructions plot different boxplots)

lc.scatter(y, markerSize = 10) + lc.column() + lc.boxplot() + lc.line()  + lc.column(x) + lc.boxplot()|> display

image

Things would be better with names for axis, titles and such. I’m not including them for the sake of simplicity.

lc.scatter(y, markerSize = 10) + lc.column() + (lc.line(x)  + lc.column()) + lc.scatter(markerSize = 20) |> display

image

If you remember the previous post, we talked about boxplots and how you generally want to have more than one on a graph. You can do that with the ‘+’ operator and get this ugly chart. More work is needed here.

lc.boxplot(y = y, xname = "Players", yname = "Ratings", title = "Players' Ratings", color = Color.Blue, whiskerPercentile = 5, percentile = 30,
    showAverage = false, showMedian = false, showUnusualValues = true) +  lc.boxplot(y = x) |> display
image 

‘++’ allows you to create new charts below the chart you already have as in:

let h = [1.;2.5;3.1;4.;4.8;6.0;7.5;8.;9.1;15.]
let w = h |> List.map (fun h -> h * 1.2)

lc.line(h) + lc.column() ++ lc.line(w) ++ lc.bubble() |> display

image

Notice the left to right flowing of information here as well. In the next installment we’ll take a look at how things are implemented and why it’s all wrong.

LChart: displaying charts in F# – Part I

I want to use F# as a exploratory data analysis language (like R). But I don’t know how to get the same nice graphic capabilities. So I decided to create them. Here is a library to draw charts in F#. It steals ideas from this book and this R package. It is nothing more than a wrapper on top of the Microsoft Chart Controls to give it a more ‘exploratory’ one line calling syntax. It is also rough work in progress: I don’t wrap all the chart types and there are bugs in the ones I wrap. Also the architecture is all wrong (more on this in another post). But it’s a start and it kind of works. Attached the full code.


I will continue this series in my new blog at wordpress: https://lucabolognese.wordpress.com/. The reason I need a new blog will be explained in an upcoming post.


Part II is now here.


Ok, let’s start. How do I draw a chart?

let x = [1.;2.5;3.1;4.;4.8;6.0;7.5;8.;9.1;15.]
let y = [1.6;2.1;1.4;4.;2.3;1.9;2.4;1.4;5.;2.9]

lc.scatter(x, y) |> display


X and Y are just some make up data. lc is the name of a class (????) and scatter is a static method on it. scatter doesn’t display the chart, it just produces a an object that represents the chart. Display displays the chart. The reason for using the bizarre lc static class is that I want it to be short so that it is easy to type in the fsi.exe. At the same time it needs to support optional parameters (which are not supported on top level functions in F#).


You get a window with this chart on it. You can press CTRL+C to copy it (as I did to post it here).


image


You might want to customize the chart a bit by passing some of these famous optional parameters:

lc.scatter(x = x, y = y, markerSize = 10, markerStyle = MarkerStyle.Diamond,
xname = “Players”, yname = “Ratings”, title = “Players’ Ratings”) |> display

image


Or you might want to print different types of charts:

lc.line(y = y, markerSize = 10, markerStyle = MarkerStyle.Diamond, xname = “Players”, yname = “Ratings”, title = “Players’ Ratings”, isValueShownAsLabel = true,
color = Color.Red) |> display
image
lc.spline(x = x, y = y, markerSize = 10, markerStyle = MarkerStyle.Diamond, xname = “Players”, yname = “Ratings”,
title = “Players’ Ratings”, isValueShownAsLabel = true, color = Color.Red) |> display
image
lc.stepline(x = x, y = y, markerSize = 10, markerStyle = MarkerStyle.Diamond, xname = “Players”, yname = “Ratings”,
title = “Players’ Ratings”, isValueShownAsLabel = true, color = Color.Red) |> display
image
lc.bar(y = y, xname = “Players”, yname = “Ratings”, title = “Players’ Ratings”, isValueShownAsLabel = true,
drawingStyle = “Emboss”) |> display
image

 

lc.column(y = y, xname = “Players”, yname = “Ratings”, title = “Players’ Ratings”,
isValueShownAsLabel = true, drawingStyle = “Cylinder”) |> display
image
lc.boxplot(y = y, xname = “Players”, yname = “Ratings”, title = “Players’ Ratings”, color = Color.Blue, whiskerPercentile = 5, percentile = 30,
showAverage = false, showMedian = false, showUnusualValues = true) |> display

image


Ok, the last one is weird. You probably want more than one boxplot in a chart. I’ll show you how to do that in the next post.


The next post will be on how to have more than one series on the same chart and more than one chart in the same windows. Something like the below:


image

ChartPlotter.fsx

A simpler F# MailboxProcessor

I always forget the pattern to use to create a functioning MailboxProcessor in F#. I mean, which piece has to be async and how to structure the recursive loop. When I find myself in that kind of a situation situation, my instincts scream at me: “Wrap it and make it work how your mind expects it to work”. So here is a simplification of the paradigm.

Let’s see how some standard MailboxProcessor code looks like:

let counter0 =
    MailboxProcessor.Start(fun inbox ->
        let rec loop n =
            async { 
                    let! msg = inbox.Receive()
                    return! loop(n+msg) }
        loop 0)

This keeps a running sum of the messages it receives. The only part that is really unique to this guy is “n + msg”. All the rest is infrastructure.

You’d probably prefer to write code like the following:

let counter1 = MailboxProcessor.SpawnAgent( (fun msg n -> msg + n), 0)

Yep, just one line of code. But, is it possible? Let’s look at one way of doing it:

type AfterError<'state> =
| ContinueProcessing of 'state
| StopProcessing
| RestartProcessing
    
type MailboxProcessor<'a> with

    static member public SpawnAgent<'b>(messageHandler :'a->'b->'b,
initialState : 'b, ?timeout:'b -> int, ?timeoutHandler:'b -> AfterError<'b>,
?errorHandler:
Exception -> 'a option -> 'b -> AfterError<'b>)
: MailboxProcessor<'a> = let timeout = defaultArg timeout (fun _ -> -1) let timeoutHandler = defaultArg timeoutHandler (fun state –>
ContinueProcessing(state)) let errorHandler = defaultArg errorHandler (fun _ _ state –>
ContinueProcessing(state)) MailboxProcessor.Start(fun inbox -> let rec loop(state) = async { let! msg = inbox.TryReceive(timeout(state)) try match msg with | None -> match timeoutHandler state with | ContinueProcessing(newState) ->
return!
loop(newState) | StopProcessing -> return () | RestartProcessing -> return! loop(initialState) | Some(m) -> return! loop(messageHandler m state) with | ex -> match errorHandler ex msg state with | ContinueProcessing(newState) -> return! loop(newState) | StopProcessing -> return () | RestartProcessing -> return! loop(initialState) } loop(initialState))

The funny formatting is because I have to fit it in the small horizontal space of this blog. In any case, this is just a simple (?) wrapper for the MailboxProcessor pattern. The function takes two necessary parameters and two optional ones:

  • messageHandler: a function to execute when a message comes in, it takes the message and the current state as parameters and returns the new state.
  • initialState: the initial state for the MailboxProcessor
  • timeoutHandler: a function that is executed whenever a timeout occurs. It takes as a parameter the current state and returns one of ContinueProcessing(newState), StopProcessing or RestartProcessing
  • errorHandler: a function that gets call if an exception is generated inside the messageHandler function. It takes the exception, the message, the current state and returns ContinueProcessing(newState), StopProcessing or RestartProcessing

An example of how to use errorHandler to implement the CountingAgent in the Expert F# book follows:

type msg = Increment of int | Fetch of AsyncReplyChannel<int> | Stop

exception StopException

type CountingAgent() =
    let counter = MailboxProcessor.SpawnAgent((fun msg n ->
                    match msg with
                    | Increment m ->  n + m
                    | Stop -> raise(StopException)
                    | Fetch replyChannel ->
                        do replyChannel.Reply(n)
                        n
                  ), 0, errorHandler = (fun _ _ _ -> StopProcessing))
    member a.Increment(n) = counter.Post(Increment(n))
    member a.Stop() = counter.Post(Stop)
    member a.Fetch() = counter.PostAndReply(fun replyChannel -> Fetch(replyChannel))    
        
let counter2 = CountingAgent()
counter2.Increment(1)
counter2.Fetch()
counter2.Increment(2)
counter2.Fetch()
counter2.Stop()                             

Sometimes your agent doesn’t need a state, it is purely stateless. Something as simple as the following:

let echo = MailboxProcessor<_>.SpawnWorker(fun msg -> printfn "%s" msg)

You can easily make that happen by using this toned down version of an agent, called worker:

static member public SpawnWorker(messageHandler,  ?timeout, ?timeoutHandler,?errorHandler) =
    let timeout = defaultArg timeout (fun () -> -1)
    let timeoutHandler = defaultArg timeoutHandler (fun _ -> ContinueProcessing(()))
    let errorHandler = defaultArg errorHandler (fun _ _ -> ContinueProcessing(()))
    MailboxProcessor.SpawnAgent((fun msg _ -> messageHandler msg; ()),
(), timeout, timeoutHandler,
(fun ex msg _ -> errorHandler ex msg))

Given that they are parallel, you might want to run a whole bunch of them at the same time. You might want something that looks like a worker, but that, under the cover, execute each messageHandler in parallel. Something like:

type msg1 = Message1 | Message2 of int | Message3 of string
            
let a = MailboxProcessor.SpawnParallelWorker(function
                | Message1 -> printfn "Message1";
                | Message2 n -> printfn "Message2 %i" n;
                | Message3 _ -> failwith "I failed"
                , 10
                , errorHandler = (fun ex _ -> printfn "%A" ex; ContinueProcessing()))


a.Post(Message1)
a.Post(Message2(100))
a.Post(Message3("abc"))
a.Post(Message2(100))

In this example, the different messages, are likely to cause things to print out of order. Notice the number 10 above which is how many workers you want to process your messages. This is implemented by round-robin messages to the various workers:

static member public SpawnParallelWorker(messageHandler, howMany, ?timeout,
?timeoutHandler,?errorHandler) = let timeout = defaultArg timeout (fun () -> -1) let timeoutHandler = defaultArg timeoutHandler (fun _ -> ContinueProcessing(())) let errorHandler = defaultArg errorHandler (fun _ _ -> ContinueProcessing(())) MailboxProcessor<'a>.SpawnAgent((fun msg (workers:MailboxProcessor<'a> array, index) -> workers.[index].Post msg (workers, (index + 1) % howMany)) , (Array.init howMany
(fun _ -> MailboxProcessor<'a>.SpawnWorker(
messageHandler, timeout, timeoutHandler,
errorHandler)), 0))

One drawback with the current code is that it doesn’t supports cancellations. It should be possible to wrap that too, but I haven’t done it (yet). If you don’t want to cut and paste the code, it is inside the AgentSystem.fs file here.