Store data to Cloud storage In GAE Managed VMs

352 views Asked by At

Environment is running app in GAE's Managed VMs.

When I run shell : gcloud preview app run build/exploded-app

And tried to upload a file and store data of the file to cloud storage:

 private final GcsService gcsService = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
      @SuppressWarnings("resource")
      GcsOutputChannel outputChannel = gcsService.createOrReplace(fileName, GcsFileOptions.getDefaultInstance());
      outputChannel.write(ByteBuffer.wrap(bytes));
      outputChannel.close();

Error happened:

com.google.appengine.tools.cloudstorage.RetriesExhaustedException: RetryHelper(5.619 s, 10 attempts, com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$1@721b2ad5): Too many failures, giving up
    at com.google.appengine.tools.cloudstorage.RetryHelper.doRetry(RetryHelper.java:127)
    at com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:166)
    at com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:156)
    at com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.close(GcsOutputChannelImpl.java:198)
    at java.nio.channels.Channels$1.close(Channels.java:178)
    at java.io.ObjectOutputStream$BlockDataOutputStream.close(ObjectOutputStream.java:1827)
    at java.io.ObjectOutputStream.close(ObjectOutputStream.java:741)
    at com.deepnet.idigisign.services.google.impl.GoogleCloudStorageServiceImpl.writeObjectToFile(GoogleCloudStorageServiceImpl.java:55)
    at com.deepnet.idigisign.services.google.impl.GoogleCloudStorageServiceImpl.write(GoogleCloudStorageServiceImpl.java:123)
    at com.deepnet.idigisign.services.document.impl.DocumentServiceImpl.setTemplateDocuments(DocumentServiceImpl.java:148)
    at com.deepnet.idigisign.services.template.impl.TemplateServiceImpl.postTemplateDocuments(TemplateServiceImpl.java:114)
    at com.deepnet.idigisign.rest.template.impl.TemplateProviderImpl.postTemplateDocuments(TemplateProviderImpl.java:101)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
    at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
    at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
    at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
    at com.deepnet.idigisign.filters.ApiOriginFilter.doFilter(ApiOriginFilter.java:25)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.deepnet.idigisign.filters.OauthOriginFilter.doFilter(OauthOriginFilter.java:49)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.deepnet.idigisign.filters.UserIdentificationServiceFilter.doFilter(UserIdentificationServiceFilter.java:42)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.googlecode.objectify.ObjectifyFilter.doFilter(ObjectifyFilter.java:48)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
    at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at com.google.apphosting.runtime.jetty9.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
    at com.google.apphosting.vmruntime.jetty9.VmRuntimeWebAppContext.doScope(VmRuntimeWebAppContext.java:461)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:497)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
    at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.google.appengine.api.files.FinalizationException
    at com.google.appengine.api.files.FileServiceImpl.translateException(FileServiceImpl.java:617)
    at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:594)
    at com.google.appengine.api.files.FileServiceImpl.open(FileServiceImpl.java:527)
    at com.google.appengine.api.files.FileServiceImpl.openForAppend(FileServiceImpl.java:471)
    at com.google.appengine.api.files.FileServiceImpl.openForAppend(FileServiceImpl.java:190)
    at com.google.appengine.api.files.FileServiceImpl.openWriteChannel(FileServiceImpl.java:171)
    at com.google.appengine.tools.cloudstorage.dev.LocalRawGcsService.append(LocalRawGcsService.java:181)
    at com.google.appengine.tools.cloudstorage.dev.LocalRawGcsService.finishObjectCreation(LocalRawGcsService.java:245)
    at com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$1.call(GcsOutputChannelImpl.java:201)
    at com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$1.call(GcsOutputChannelImpl.java:198)
    at com.google.appengine.tools.cloudstorage.RetryHelper.doRetry(RetryHelper.java:108)
    ... 76 more
Caused by: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 101: File is already finalized
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.convertRemoteError(VmApiProxyDelegate.java:276)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.runSyncCall(VmApiProxyDelegate.java:174)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.makeApiCall(VmApiProxyDelegate.java:141)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.makeSyncCallWithTimeout(VmApiProxyDelegate.java:115)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.makeSyncCall(VmApiProxyDelegate.java:104)
    at com.google.apphosting.vmruntime.VmApiProxyDelegate.makeSyncCall(VmApiProxyDelegate.java:47)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:112)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:64)
    at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:590)
    ... 85 more

Can someone tell me where it all went wrong?

1

There are 1 answers

3
ozarov On

AE gcs client should work fine on managed VMs but unfortunately its development (fake) service (which seems to be used in your stack-trace - LocalRawGcsService) is still using the deprecated files API. This only effects development environment and there is a work in progress to fix it.