I was investigating an F# script running via FsiAnyCpu which was using a lot of memory. I took a heap snapshot using PerfView and it looks like the majority of high count nodes were rooted in FSharp.Compiler!Microsoft.FSharp.Compiler.Ilxgen+IlxAssemblyGenerator. Before taking the snapshot, I forced a GC and froze the process for the duration.
The default exclusive view shows:
Is this to be expected?
FSI generates assemblies to contain code that you evaluate. That code can hold rooted references to heap objects. Expressions that you evaluate can cause rooted references as well. FSI is likely to behave as a by-design memory leak--if a REPL didn't remember what you'd previously evaluated it might not be so useful. It can also hold open references to files and assemblies, for the same reasons.
Within Visual Studio FSI provides a context menu command called "Reset Interactive Session" which cleans everything up. For the command line FSI presumably you can just terminate it.