composing many quotations into linq queries

Question

I'm working on a project in which I'm trying to use F# and Linq for UDF's and stored procs in an SQL server. Part of that has been to statically define all the valid queries, the sorting criteria, and a means of scoring the results of the queries.

I've so far been fairly successful, but I'm running into serious difficulty composing sortBy expressions.

Here's the basic concept

let sorter =
    let exprMap:Map<string,Quotations.Expr<seq<Product> -> seq<Product>>> =
    Map.ofList
    ["ProductName",<@ Seq.sortBy (fun prod -> prod.Name) @> ]
    // .. more entries ..
    let sortBuilder sortkeys = 
        Array.foldBack 
         (fun criteria acc -> <@ %(exprMap.[criteria]) >> (%acc) @>)
         sortkeys
         <@ Seq.map id @>

This ends up being used later in the query executor like so

let execQuery = fun (predicates,sorts,scorer) ->
    <@ seq { for prod in (%dc).Products do
              if (%predicates) prod then yield prod }
       |> (%sorts)
       |> (%scorer) @> 

Using these basic outlines, everything works as long as I don't use (%sorts). Each time I pass that in, I get not recognized in F# to Linq translator. I've tried a number of different attempts at using combinators, but I have the sense I'm missing something. If I stub out the sorter function with the following

<@ Seq.sortBy (fun prod -> prod.Name) |> Seq.sortBy (fun prod -> prod.Style) @>

It works as expected. However using a combinator like this:

let (|>*) = fun f g -> <@ fun c -> ((%f) c) |> (%g) @>

does not..

Any ideas?

Answer 1

Unfortunately, I don't have any good answer to this question.

I'm afraid that the F# LINQ translator is currently very sensitive to the structure of the query. Using composition, you should be able to get the same quotation you get if you write it by hand, so you may need to generate exactly the same thing that worked if written by hand.

For example with your sorter, you may need something like (I didn't try it, but I think this should produce exactly the same quotation as the usual code that works):

let (|>*) f g = fun c -> <@ (%c) |> (%f) |> (%g) @> 

<@ seq { for prod in (%dc).Products do 
           if (%predicates) prod then yield prod } @> |>
( <@ Seq.sortBy (fun prod -> prod.Name) @> |>*
  <@ Seq.sortBy (fun prod -> prod.Style) @> )

The problem is that if you include lambda functions in the quotation, the F# translator needs to deal with them - probably by partially evaluating them (because otherwise the LINQ to SQL translator would fail). There are quite a few tricky cases in this...

However, the F# team has been doing some improvements in this area recently. I think the best thing to do would be to find a simple repro case and send it to fsbugs at microsoft dot com. PowerPack releases are not that "sensitive" so you may be able to get the source code with the recent changes if you ask and offer help with testing (but no promises).