Copy Production Cloud Datastore to Dev Server Datastore

988 views Asked by At

According to the online documentation, in order to backup data you must go to the Datastore Admin after you have enabled it. It is enabled on my production server.

The to import you simply go to the admin page again, and select the back name you just created. I have downloaded the backup from my Google Bucket onto a local drive, following all the steps of this handy guide found here http://gbayer.com/big-data/app-engine-datastore-how-to-efficiently-export-your-data/

Now that I have the data locally, I am stuck in trying to get it into a local dev server.

appcfg.py upload_data --application=dev~<APP_ID> --url=http://localhost:8080/_ah/remote_api --filename=<lengthyfilename>.backup_info
04:03 PM Uploading data records.
[INFO    ] Logging to bulkloader-log-20161230.160323
[INFO    ] Throttling transfers:
[INFO    ] Bandwidth: 250000 bytes/second
[INFO    ] HTTP connections: 8/second
[INFO    ] Entities inserted/fetched/modified: 20/second
[INFO    ] Batch Size: 10
[INFO    ] Opening database: bulkloader-progress-20161230.160323.sql3
[ERROR   ] [Thread-1] RestoreThread:
Traceback (most recent call last):
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\bulkloader.py", line 1555, in run
    self.PerformWork()
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\bulkloader.py", line 2838, in PerformWork
    cursor.execute('select id, value from result')
DatabaseError: file is encrypted or is not a database

Inside the local admin page, there is not a restore option in any of the Cloud Datastore pages, I'm at a loss. All the answers I find are five years old, on outdated packages. How do you restore an Cloud Datastore Backup into a Developer App Server?

1

There are 1 answers

0
Juan Lara On

One benefit of using the Datastore emulator is that it now supports import and export.

You can read about starting dev_appserver with the Datastore emulator here.