CStdioFile cannot work with files larger than 2GB?

Question

I am using Visual C++ 2008. In VC++ 2008, CFile supports 2^64 huge files. So I think CStdioFile should also support.

However, when using CStdioFile::GetLength() on a file larger than 2GB, I get a CFileException, below is the code snippet:

void CTestCStdioFileDlg::OnBnClickedButton1()
{
    // TODO: Add your control notification handler code here
    CStdioFile MyFile;
    CString strLine;
    ULONGLONG uLength;

    strLine = _T("This is a line.");

    if (MyFile.Open(_T("C:\\Temp\\MyTest.dat"), CFile::modeCreate | CFile::modeWrite | CFile::shareExclusive | CFile::typeBinary))
    {
        for (UINT uIndex = 0; uIndex = 200000000; uIndex ++)
        { 
            MyFile.WriteString(strLine);
            uLength = MyFile.GetLength();
        }

        MyFile.Close();
    }
}

After tracing into the CStdio::GetLength(), I find the following code snippet will raise exception, as below:

   nCurrent = ftell(m_pStream);            -> This will return -1
   if (nCurrent == -1)
      AfxThrowFileException(CFileException::invalidFile, _doserrno,
         m_strFileName);

It is amazing that CStdioFile still use ftell instead of _ftelli64 to deal with the stream.

I then search the document for CStdioFile, I cannot find any document on CStdioFile::GetLength, the only related is https://learn.microsoft.com/en-us/cpp/mfc/reference/cstdiofile-class?view=vs-2019#seek, and it asks me to see fseek document. But in fseek document, I still not find any related to file size limit.

Finally I find a third-party site that indicates CStdioFile::GetLength contains error: http://www.flounder.com/msdn_documentation_errors_and_omissions.htm#CStdioFile::GetLength, but it does not provide a solution.

Other than these, there are hardly any questions or posts on the 2GB limit of CStdioFile online. That is really strange.

I try to check the source code of CStdioFile iN VC++ 2017 and it is same as that for 2008.

So whether there is a simple solution for the problem without rewriting the whole CStdioFile class?

Answer 1

CStdioFile exists primarily for a single reason: It's constructor taking a FILE* argument. The purpose of this class is to wrap a C Runtime file and expose it through a CFile-compatible interface. The implementation inherits all of the C Runtime limitations, in particular the file size limit of 2GB.

To address this, there are several options:

1. If possible, drop the use of CStdioFile altogether. Unless you are interfacing with (legacy) C code, there's no striking reason to ever use it.

2. If that is not an option, derive a custom implementation from CStdioFile and override all class members that surface file-relative offsets (GetPosition(), GetLength(), Seek()). All other class members are unaffected, and can simply be inherited. (Note: Make sure to restore the current file pointer when implementing GetLength().)

   Microsoft provides extensions to its C Runtime that offer 64-bit wide offsets (_ftelli64 and _fseeki64).

If you need to go with option 2, the following extension to CStdioFile can be used as a drop-in replacement with support for files larger than 2GB.

CStdioFileExt.h:

#pragma once

#include <afx.h>

class CStdioFileExt : public CStdioFile
{
    DECLARE_DYNAMIC(CStdioFileExt)
public:
    ULONGLONG GetPosition() const override;
    ULONGLONG GetLength() const override;
    ULONGLONG Seek(LONGLONG lOff, UINT nFrom) override;
};

CStdioFileExt.cpp:

#include "CStdioFileExt.h"

ULONGLONG CStdioFileExt::GetPosition() const
{
    ASSERT_VALID(this);
    ASSERT(m_pStream != NULL);

    auto const pos = _ftelli64(m_pStream);
    if (pos == -1L)
        AfxThrowFileException(CFileException::invalidFile, _doserrno, m_strFileName);
    return static_cast<ULONGLONG>(pos);
}

ULONGLONG CStdioFileExt::GetLength() const
{
    ASSERT_VALID(this);

    auto const nCurrent = _ftelli64(m_pStream);
    if (nCurrent == -1L)
        AfxThrowFileException(CFileException::invalidFile, _doserrno, m_strFileName);

    auto nResult = _fseeki64(m_pStream, 0, SEEK_END);
    if (nResult != 0)
        AfxThrowFileException(CFileException::badSeek, _doserrno, m_strFileName);

    auto const nLength = _ftelli64(m_pStream);
    if (nLength == -1L)
        AfxThrowFileException(CFileException::invalidFile, _doserrno, m_strFileName);
    nResult = _fseeki64(m_pStream, nCurrent, SEEK_SET);
    if (nResult != 0)
        AfxThrowFileException(CFileException::badSeek, _doserrno, m_strFileName);

    return static_cast<ULONGLONG>(nLength);
}

ULONGLONG CStdioFileExt::Seek(LONGLONG lOff, UINT nFrom)
{
    ASSERT_VALID(this);
    ASSERT(nFrom == begin || nFrom == end || nFrom == current);
    ASSERT(m_pStream != NULL);

    if (_fseeki64(m_pStream, lOff, nFrom) != 0)
        AfxThrowFileException(CFileException::badSeek, _doserrno, m_strFileName);

    auto const pos = _ftelli64(m_pStream);
    return static_cast<ULONGLONG>(pos);
}

IMPLEMENT_DYNAMIC(CStdioFileExt, CStdioFile)