Stream YAML output, rather than loading everything into memory

46 views Asked by At

I am using Perl with YAML::XS to write YAML output. Specifically, this YAML output is a list of the contents of other YAML files. I'm doing so, for example, like this:

use YAML::XS qw(LoadFile DumpFile);

my @yaml_files = qw(foo.yaml bar.yaml quux.yaml);

DumpFile("output.yaml", [
   map { LoadFile($_) } @yaml_files
]);

My understanding is that this will load every input YAML file into memory to create the list, then finally write the output to disk. In general, that makes sense, but in this case, there's an opportunity to stream the output one list item at a time. How can this be done, if it's indeed possible, with YAML::XS?


Notes:

  • I could do some string manipulation myself, to stream the list output -- rather than using DumpFile -- but that relies on me outputting correct YAML. It's relatively trivial in this case, but I'd rather defer to library code.
  • My environment is pretty constrained. I have YAML::XS, but that's it beyond the standard library.
0

There are 0 answers