Building my unaltered package suddenly fails: for Vignettes and 'arch = x64'

Question

Context

I've been developing a proprietary package (call it "my_pkg") with vignettes, in RStudio with the latest version (4.1.1) of R. I'm working on a Lenovo ThinkPad with the following software

       sysname        release        version       machine
     "Windows"       "10 x64"  "build 19043"      "x86-64"

as provided by Sys.info().

Until recently, my devtools workflow has been reasonably smooth (with one occasional exception). However, I've begun to encounter a very puzzling error, which occurs under conditions virtually identical to those beforehand, when the workflow proceeded without issue.

Despite the fact that my vignettes always built successfully before, and I have not edited them in the meantime, the build now fails for architecturally specific reasons...

Error : package 'my_pkg' is not installed for 'arch = x64'

...when and only when I include the vignettes in the build.

I have reinstalled R (and Rtools and RStudio) from scratch, reinstalled devtools (and its dependencies), and "rewound" my project to the last Git commit at which devtools::check() succeeded. I have also walked through extensive troubleshooting with responders like @SteffenMoritz and @Alexis, for whom I have appended further information in the section Info by Request.

Unfortunately, nothing has worked.

Failure With devtools::check()

When I run devtools::check(), everything starts promisingly...

i Updating my_pkg documentation
i Loading my_pkg
Writing NAMESPACE
Writing NAMESPACE
-- Building --------------------------------------------------------------------------------- my_pkg --
Setting env vars:
* CFLAGS    : -Wall -pedantic
* CXXFLAGS  : -Wall -pedantic
* CXX11FLAGS: -Wall -pedantic
-------------------------------------------------------------------------------------------------------
√  checking for file 'C:\Users\greg\Workspace\R\Packages\my_pkg/DESCRIPTION' ...
-  preparing 'my_pkg': (733ms)
√  checking DESCRIPTION meta-information ... 
-  installing the package to build vignettes
         -----------------------------------
-  installing *source* package 'my_pkg' ...
   ** using staged installation
   ** R
   ** inst
   ** byte-compile and prepare package for lazy loading
   ** help
   *** installing help indices
     converting help for package 'my_pkg'
       finding HTML links ... done 
       foo                                     html
       my_pkg-package                          html
   *** copying figures
   ** building package indices
   ** installing vignettes
   ** testing if installed package can be loaded from temporary location

---

...until the step testing if installed package can be loaded from temporary location. Here an error occurs:

   Error : package 'my_pkg' is not installed for 'arch = x64'
   Error: loading failed
   Execution halted
   ERROR: loading failed
-  removing 'C:/Users/greg/AppData/Local/Temp/RtmpSaabgx/Rinst332419517258/my_pkg'
         -----------------------------------
   ERROR: package installation failed
Error in (function (command = NULL, args = character(), error_on_status = TRUE,  : 
  System command 'Rcmd.exe' failed, exit status: 1, stdout + stderr (last 10 lines):
E> ** building package indices
E> ** installing vignettes
E> ** testing if installed package can be loaded from temporary location
E> Error : package 'my_pkg' is not installed for 'arch = x64'
E> Error: loading failed
E> Execution halted
E> ERROR: loading failed
E> * removing 'C:/Users/greg/AppData/Local/Temp/RtmpSaabgx/Rinst332419517258/my_pkg'
E>       -----------------------------------
E> ERROR: package installation failed
Type .Last.error.trace to see where the error occurred

The .Last.error.trace suggests an issue with pkgbuild specifically:

> .Last.error.trace

 Stack trace:

 1. devtools::check()
 2. withr::with_envvar(pkgbuild::compiler_flags(FALSE), action = "prefix",  ...
 3. base:::force(code)
 4. pkgbuild::build(pkg$path, tempdir(), args = build_args, quiet = quiet,  ...
 5. withr::with_temp_libpaths(rcmd_build_tools(options$cmd, c(options$path,  ...
 6. base:::force(code)
 7. pkgbuild:::rcmd_build_tools(options$cmd, c(options$path, options$args),  ...
 8. pkgbuild:::with_build_tools(callr::rcmd_safe(..., env = env,  ...
 9. withr::with_path(rtools_path(), code)
 10. base:::force(code)
 11. callr::rcmd_safe(..., env = env, spinner = FALSE, show = FALSE,  ...
 12. callr:::run_r(options)
 13. base:::with(options, with_envvar(env, do.call(processx::run,  ...
 14. base:::with.default(options, with_envvar(env, do.call(processx::run,  ...
 15. base:::eval(substitute(expr), data, enclos = parent.frame())
 16. base:::eval(substitute(expr), data, enclos = parent.frame())
 17. callr:::with_envvar(env, do.call(processx::run, c(list(bin, args = real_cmdargs,  ...
 18. base:::force(code)
 19. base:::do.call(processx::run, c(list(bin, args = real_cmdargs,  ...
 20. (function (command = NULL, args = character(), error_on_status = TRUE,  ...
 21. throw(new_process_error(res, call = sys.call(), echo = echo,  ...

---

I have no clue as to why the architecture

Error : package 'my_pkg' is not installed for 'arch = x64'

should be an issue only now. There have been no substantive changes to my computer.

Despite the message package installation failed, and despite the fact that I've run devtools::check() rather than devtools::install(), the package acts as if it were "halfway" installed! I can type my_p in the console, and RStudio will autocomplete my_pkg; and I can then type my_pkg:: and RStudio will supply the standard autocompletion list of the functions in my_pkg, which are all available.

However, typing my_pkg:: not only summons the @exported functions but also exposes my internal functions and data, as if I had typed my_pkg:::!

Success With devtools::install() or devtools::build(vignettes = FALSE)

Oddly, when I actually run devtools::install(), or comparably devtools::build_vignettes(vignettes = FALSE), the process completes without a hitch!

√  checking for file 'C:\Users\greg\Workspace\R\Packages\my_pkg/DESCRIPTION' ...
-  preparing 'my_pkg': (640ms)
√  checking DESCRIPTION meta-information ... 
-  checking for LF line-endings in source and make files and shell scripts (384ms)
-  checking for empty or unneeded directories
   Removed empty directory 'my_pkg/inst/libs'
   Omitted 'LazyData' from DESCRIPTION
-  building 'my_pkg_0.0.0.9000.tar.gz'
   
Running "C:/Users/greg/R/bin/x64/Rcmd.exe" INSTALL \
  "C:\Users\greg\AppData\Local\Temp\Rtmpozvh26/my_pkg_0.0.0.9000.tar.gz" --install-tests 
* installing to library 'C:/Users/greg/R/library'
* installing *source* package 'my_pkg' ...
** using staged installation
** R
** inst
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
  converting help for package 'my_pkg'
    finding HTML links ... done
       foo                                     html
       my_pkg-package                          html
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
*** arch - i386
*** arch - x64
** testing if installed package can be loaded from final location
*** arch - i386
*** arch - x64
** testing if installed package keeps a record of temporary installation path
* DONE (my_pkg)
Making 'packages.html' ... done

Now, the autocompletion works entirely as expected, and only the appropriate functions are exposed by my_pkg:: and my_pkg::: respectively.

Failure With devtools::install(build_vignettes = TRUE) or devtools::build()

Yet there is one further plot twist. When I use devtools::install(build_vignettes = TRUE), or comparably devtools::build(), I get a similar error to that with devtools::check():

√  checking for file 'C:\Users\greg\Workspace\R\Packages\my_pkg/DESCRIPTION' ...
-  preparing 'my_pkg': (672ms)
√  checking DESCRIPTION meta-information ... 
-  installing the package to build vignettes
         -----------------------------------
-  installing *source* package 'my_pkg' ...
   ** using staged installation
   ** R
   ** inst
   ** byte-compile and prepare package for lazy loading
   ** help
   *** installing help indices
     converting help for package 'my_pkg'
       finding HTML links ... done
       foo                                     html
       my_pkg-package                          html
   *** copying figures
   ** building package indices
   ** installing vignettes
   ** testing if installed package can be loaded from temporary location

---

   Error : package 'my_pkg' is not installed for 'arch = x64'
   Error: loading failed
   Execution halted
   ERROR: loading failed
-  removing 'C:/Users/greg/AppData/Local/Temp/RtmpM7RbHr/Rinst504119422cd/my_pkg'
         -----------------------------------
   ERROR: package installation failed
Error in (function (command = NULL, args = character(), error_on_status = TRUE,  : 
  System command 'Rcmd.exe' failed, exit status: 1, stdout + stderr (last 10 lines):
E> ** building package indices
E> ** installing vignettes
E> ** testing if installed package can be loaded from temporary location
E> Error : package 'my_pkg' is not installed for 'arch = x64'
E> Error: loading failed
E> Execution halted
E> ERROR: loading failed
E> * removing 'C:/Users/greg/AppData/Local/Temp/RtmpM7RbHr/Rinst504119422cd/my_pkg'
E>       -----------------------------------
E> ERROR: package installation failed
Type .Last.error.trace to see where the error occurred

Success With devtools::build_vignettes()

Bizarrely, despite the issues above with building vignettes, everything somehow runs smoothly when I call devtools::build_vignettes() sans arguments or with dependencies = TRUE:

i Building my_pkg vignettes
√ Creating 'doc/'
√ Adding '^doc$' to '.Rbuildignore'
√ Setting active project to '<no active project>'
√ Adding '/doc/' to '.gitignore'
√ Setting active project to '<no active project>'
i Moving backstory.html, intro.html, library.html, template.html, usage.html, backstory.R, intro.R, library.R, template.R, and usage.R to doc/
i Copying backstory.Rmd, intro.Rmd, library.Rmd, template.Rmd, and usage.Rmd to doc/
√ Creating 'Meta/'
√ Adding '^Meta$' to '.Rbuildignore'
√ Setting active project to '<no active project>'
√ Adding '/Meta/' to '.gitignore'
√ Setting active project to '<no active project>'
i Building vignette index

Note

Assuming that the fault was mine, and that I had unwittingly introduced an issue somewhere within my R project, I switched to an entirely different Git branch. The most recent commit to this branch had passed devtools::check() with flying colors. However, even after switching over to this "safe" branch, those same issues continued!

Between that last successful check and when the issues began, I made no updates to:

• R itself
• RStudio
• Any R packages
• My computer's software (with the exception of a minor anti-malware update from Microsoft).
• My computer's hardware

Likewise, the vignettes have been present since far earlier in the project.

Conclusion

I'm hoping to roll out a proof of concept for my_pkg in the rather near future, and the vignettes are quite essential to its usability.

---

Info by Request

Since "my_pkg" is proprietary, I here anonymize my personal and organizational information (names, filepaths, etc.) as above.

For @SteffenMoritz

Would you mind sharing your DESCRIPTION file and the vignette header/meta information?

The DESCRIPTION file:

Package: my_pkg
Title: Proprietary Package for My Org
Version: 0.0.0.9000
Authors@R: 
    person(given = "Greg",
           family = "Lastname",
           role = c("aut", "cre"),
           email = c("[email protected]"))
Description: The `my_pkg` package is a proprietary project for My Org,
    and as such I have changed the names to project the innocent.
License: file LICENSE
Encoding: UTF-8
LazyData: true
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.2
URL: https://github.com/myorg/my_pkg
BugReports: https://github.com/my_org/my_pkg/issues
Suggests: 
    knitr,
    rmarkdown,
    testthat (>= 3.0.0)
Config/testthat/edition: 3
VignetteBuilder: knitr
Depends: 
    R (>= 2.10)

The vignette headers:

• intro.Rmd:

---
title: "A Catchphrase!"
author: "Gregory Lastname"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{A Catchphrase!}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  # Show no output.
  eval = FALSE, results = "hide",
  # Show no dialog.
  message = FALSE, warning = FALSE,
  # Show no errors.
  error = TRUE, purl = FALSE,
  # Default format.
  collapse = TRUE, comment = "#>"
)
    ```

• backstory.Rmd:

---
title: "A Reference, to Science Fiction"
author: "Gregory Lastname"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{A Reference, to Science Fiction}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  # Show no output.
  eval = FALSE, results = "hide",
  # Show no dialog.
  message = FALSE, warning = FALSE,
  # Show no errors.
  error = TRUE, purl = FALSE,
  # Default format.
  collapse = TRUE, comment = "#>"
)
    ```

• library.Rmd

---
title: "A Generic One-Liner"
author: "Gregory Lastname"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{A Generic One-Liner}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  # Show no output.
  eval = FALSE, results = "hide",
  # Show no dialog.
  message = FALSE, warning = FALSE,
  # Show no errors.
  error = TRUE, purl = FALSE,
  # Default format.
  collapse = TRUE, comment = "#>"
)
    ```

• template.Rmd

---
title: "A Well-Written Description"
author: "Gregory Lastname"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{A Well-Written Description}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  # Show no output.
  eval = FALSE, results = "hide",
  # Show no dialog.
  message = FALSE, warning = FALSE,
  # Show no errors.
  error = TRUE, purl = FALSE,
  # Default format.
  collapse = TRUE, comment = "#>"
)
    ```

• usage.Rmd

---
title: "An Exhortation!"
author: "Gregory Lastname"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{An Exhortation!}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  # Show no output.
  eval = FALSE, results = "hide",
  # Show no dialog.
  message = FALSE, warning = FALSE,
  # Show no errors.
  error = TRUE, purl = FALSE,
  # Default format.
  collapse = TRUE, comment = "#>"
)
    ```

For @Alexis

What do you see if you open the .Rproj file with a text editor?

The my_pkg.Rproj file, as opened in MS Notepad. NOTE: This file has not been modified since the very initial commit.

Version: 1.0

RestoreWorkspace: No
SaveWorkspace: No
AlwaysSaveHistory: Default

EnableCodeIndexing: Yes
UseSpacesForTab: Yes
NumSpacesForTab: 2
Encoding: UTF-8

RnwWeave: Sweave
LaTeX: pdfLaTeX

AutoAppendNewline: Yes
StripTrailingWhitespace: Yes
LineEndingConversion: Posix

BuildType: Package
PackageUseDevtools: Yes
PackageInstallArgs: --no-multiarch --with-keep.source
PackageRoxygenize: rd,collate,namespace

By request

Does it work if you run R CMD build/INSTALL ... from the command line?

I also built from the command line

greg@CO-GLASTNAME MINGW64 ~/Workspace/R
$ R CMD build C:/Users/greg/Workspace/R/Packages/my_pkg

but I got the same output echoed above:

* checking for file 'C:/Users/greg/Workspace/R/Packages/my_pkg/DESCRIPTION' ... OK
* preparing 'my_pkg':
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
      -----------------------------------
* installing *source* package 'my_pkg' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
  converting help for package 'my_pkg'
    finding HTML links ... done
    foo                                     html
    my_pkg-package                          html
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error : package 'my_pkg' is not installed for 'arch = x64'
Error: loading failed
Execution halted
ERROR: loading failed
* removing 'C:/Users/greg/AppData/Local/Temp/RtmpgFIc2r/Rinst1cc0231a6b37/my_pkg'
      -----------------------------------
ERROR: package installation failed

Answer 1

You are probably right and it is a problem with the vignette. (but of course I can't be 100% sure, as I don't know your files)

About your confusion about the check() and install() runs:

RStudio’s “build and reload doesn’t build vignettes

That is why you probably have to run devtools::install() to get the error.

I'll post you some most likely issues with the vignette (here is also a good longer tutorial):

First check the following things:

1. The directory is called vignettes/ and not vignette/.

2. Check that vignettes are not covered by .Rbuildignore

3. Ensure you have the necessary vignette metadata (your seemed good from the first look)

Most likely these thing will be alright for you (but just to check).

In my opinion the most likely issue is, that your DESCRIPTION file does not contain all the necessary dependencies for your .Rmd vignettes

Also better write:

VignetteBuilder: knitr, rmarkdown

Still keep rmarkdown additionally in the Suggests. Also knitr to both, Suggests and VignetteBuilder.

Even more important(!) you also need the packages, that are used in your .Rmd files in Suggests. Since you are using RMarkdown I am assuming, there is some code in your vignettes. Go through your code and look which packages you are using. Any packages used by the vignette should be declared in the DESCRIPTION.

I could imagine this might also be the reason, why it seemingly worked once and then after changing nothing (in your opinion) it fails.

Could be you still had all the required packages loaded in the background when it worked. So that it did not have an impact, that the packages were missing in Suggests.

This would be my first guess, what the problem could be. If this does not work, I would try removing vignettes, to see if it is a certain vignette that causes the problem.

Answer 2

What solved it in my case was switching from %\VignetteEngine{knitr::rmarkdown} to %\VignetteEngine{knitr::knitr}. I have no idea why, but it stopped the error and putting back rmarkdown instead of knitr reproduced the error.

Answer 3

Unresolved

As of now, no answer has been posted that solves the titular error. I am posting this by way of clarification, to any users encountering the same error, that the error remains undiagnosed and unresolved. Indeed, the error has persisted even after resetting my project via Git and reinstalling R (and Rtools and RStudio) from scratch.

My thanks to @SteffenMoritz and @Alexis for their tenacious debugging sessions. While our efforts were unsuccessful*, I appreciate their time, effort, and friendliness. Anyone struggling with similar issues would do well to follow their advice, found in an answer and in the comments.

I will update this post with any true solution I might encounter in the future.

* Hence, the bounty has been left to the default mechanism (and so halved) upon expiration.