Git CRLF LF EOL Conversion Setting

Question

Platform: Windows 8.1 Emacs 24.3

The git config --global -l shows:

user.name=username
user.email=useremail
core.autocrlf=false
core.safecrlf=true

Git repository .gitattributes file:

# Auto detect text files and perform LF normalization
* text=auto

# Custom for Visual Studio
*.cs     diff=csharp
*.sln    merge=union
*.csproj merge=union
*.vbproj merge=union
*.fsproj merge=union
*.dbproj merge=union

# Standard to msysgit
*.doc    diff=astextplain
*.DOC    diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot  diff=astextplain
*.DOT  diff=astextplain
*.pdf  diff=astextplain
*.PDF    diff=astextplain
*.rtf    diff=astextplain
*.RTF    diff=astextplain

I do think my Git and repository settings are right.

But everytime I create a new text file with Emacs, I cannot run git add <newfile>. The file is encoded by utf-8-unix.

The error message is as:

E:\workspace\repository [master +0 ~2 -0]> git add .
fatal: LF would be replaced by CRLF in newfile.txt

I don't think is due to emacs editor problem. Because I opened the new file and pretty sure the line ending is LF not the windows default CRLF.

Which configuration part decides LF will be replaced by CRLF?

---

EDIT 1
If safecrlf is set to warn the output is:

warning: LF will be replaced by CRLF in _posts/2014-11-19-test.md.
The file will have its original line endings in your working directory.

This means that the file was successfully added to the index. My file is encoded by utf-8-unix.

---

EDIT 2

Interestingly, if I create a new file with Notepad not the Emacs 24.3, the file can be added without any problem. The difference is Notepad adopts CRLF line ending while Emacs 24.3 adopts LF line ending.

So the problem is somewhere somehow Git converts CRLF to LF then back to CRLF which generates error for original LF line ending file.

---

EDIT 3

Previously, GitHub Windows GUI client warned me no .gitattributes file for my repository and recommend a default .gitattributes file as above.

I think the problem is from the line * text=auto. So I comment out this line.

Everything works good now!

---

EDIT 4

The core is:

• DISABLE AUTO LINE ENDING CONVERSION by GitHub.

• DEPEND ON PLATFORM FILE EDITOR FOR LINE ENDING.

---

EDIT 5

1. text This attribute enables and controls end-of-line normalization. When a text file is normalized, its line endings are converted to LF in the repository. To control what line ending style is used in the working directory, use the eol attribute for a single file and the core.eol configuration variable for all text files.
   • Setting the text attribute on a path enables end-of-line normalization and marks the path as a text file. End-of-line conversion takes place without guessing the content type.
   • Unsetting the text attribute on a path tells Git not to attempt any end-of-line conversion upon checkin or checkout.
   • When text is set to "auto", the path is marked for automatic end-of-line normalization. If Git decides that the content is text, its line endings are normalized to LF on checkin.
   • Unspecified. If the text attribute is unspecified (by !text or no setting entry at all), Git uses the core.autocrlf configuration variable to determine if the file should be converted (fall-back compatibility as noted in EDIT 8).
2. eol This attribute sets a specific line-ending style to be used in the working directory. It enables end-of-line normalization without any content checks, effectively setting the text attribute.
   • That is to say eol automatically set text attribute.
   • Set to string value "crlf"
     This setting forces Git to normalize line endings for this file on checkin and convert them to CRLF when the file is checked out.
   • Set to string value "lf"
     This setting forces Git to normalize line endings to LF on checkin and prevents conversion to CRLF when the file is checked out.

If eol is put in .gitattributes file, it should be applied to specific file type. At the same time, it automatically marks the specific file type as text at the same time for LF normalization when checkin. If eol is set as git config --global core.eol xxx, then eol is set for all text files.

Refer to gitattributes - defining attributes per path

---

EDIT 6

Git attributes are specified in .gitattributes files. Line endings are controlled by text and eol attributes.

text attribute tells Git whether the file is binary (i.e. no EOL conversion should be performed while checking out and in) or text (perform EOL conversion, always convert to LF while checking in). Possible values are set (EOLs conversion is turned on), unset(EOLs conversion is turned off, default value) and auto(if the file is detected as binary, no conversion, otherwise EOLs conversion is performed).

eol attribute: if set implicitly sets text attribute and defines EOL to which the file should be converted while checking out.

Refer to Line endings handling in SVN, Git and SubGit

---

EDIT 7

Possible solutions:

1. As EDIT 3, comment out * text=auto
2. Add a line in gitattributes for .md file: *.md eol=lf
   • If another kind of text file such as .txt is created, I should also add a line for it *.txt eol=lf
3. Change * text=auto to * !text
4. Totally remove the .gitattributes file

To my current knowledge I think:

1. By filetype text in gitattributes, EOL normalization is carried out for filetype at checkin.
2. AT THE SAME TIME, this line implicates that default eol should also be carried out when checkout. default means depending on global configuration as git config --global core.eol xxx or default operating system style as core.eol = native.
   • You can run git config --global core.eol to see what this value is set to on your system. If nothing comes back that means you are on the using the OS default which is native.
3. eol implicates text attribute, while text implicates default eol attribute.
4. So when *.md text=auto is enabled. *.md file is detected by Git as text file and converted to LF when checkin. When checkout, *.md LF will be converted to CRLF. This process is invalidated by safecrlf = false global setting, refusing to adding the file to index/stage area.

---

EDIT 8

My three assumptions in EDIT 7 is verified Mind the End of Your Line

Since Git 1.7.2 and above, EOL settings are mainly put in .gitattributes in the root directory of working tree. The global config autocrlf is only for fall-back compatibility.

*.txt text Set all files matching the filter *.txt to be text. This means that Git will run CRLF to LFreplacement on these files every time they are written to the object database and the reverse replacement will be run when writing out to the working directory.

---

EDIT 9 Final Idea

1. text and core.autocrlf(in .gitattributes) focuses on EOL conversion writing to repository database from working tree.
2. eol and core.eol(in global config) focuses on EOL conversion writing to working tree from repository database.
3. .gitattributes has higher precedence than global config when deciding EOL conversion. The later one is actually fall-back reference.
4. If one way conversion is specified, the other way is unspecified or undefined, the default fall-back option is used for the other way conversion.
5. The EOL issue associates closely with which line ending you choose when editing files.

For logical explanation refer to my blog znhoo

Answer 1

The culprit is the core.safecrlf=true in your Git config.

From the git-config manual page:

core.safecrlf
    If true, makes git check if converting CRLF is reversible when
    end-of-line conversion is active. Git will verify if a command
    modifies a file in the work tree either directly or indirectly. For
    example, committing a file followed by checking out the same file
    should yield the original file in the work tree. If this is not the
    case for the current setting of core.autocrlf, git will reject the
    file. The variable can be set to "warn", in which case git will only
    warn about an irreversible conversion but continue the operation.

Answer 2

Possible solutions:

1. As EDIT 3, comment out * text=auto
2. Add a line in gitattributes for .md file: *.md eol=lf
   • If another kind of text file such as .txt is created, I should also add a line for it *.txt eol=lf
3. Change * text=auto to * !text
4. Totally remove the .gitattributes file