streams, stream_bufs, codecvt facets and \n to \r\n translation

620 views Asked by At

What part of the C++ IO streams does the \r to \r\n conversion? Is it the stream_buf itself or is it part of the internal to external encoding conversion by codecvt facet?

UPDATE 1

You all say that it is done in streambuf/filebuf. Ok. But how does this arrangement deal with, e.g., external encodings like UTF-16? Then it seems that the file has to be opened with ios::binary flag which disables the translation.

3

There are 3 answers

2
Cubbi On BEST ANSWER

This conversion is not (usually) performed by stream, streambuf, or facet. It is the responsibility the C library code (e.g. fputc()) that is called by streambuf's overflow() and underflow().

If you need it for some reason (e.g. when implementing a dos2unix routine), there's a handy example in boost.iostreams.

EDIT: std::filebuf only supports multibyte encodings for text files, e.g UTF-8 or GB18030 or whatever the locale uses. A UTF-16 file would have to be opened in binary mode, as a plain byte stream (which can be interpreted as UTF-16 with C++11's codecvt facilities), and yes the line endings would not get converted.

2
πάντα ῥεῖ On

IFAIR it's done in streambuf implementation, codecvt just deals with the locale representation specifics.

1
Éric Malenfant On

It is performed by std::filebuf, if it was open without the ios::binary flag.