I've updated couchbase cluster from version 2.5.1 to 3.0.3
. Upgrade was performed node by node by removing one node from cluster, upgrading it and add it again.
All was working but not the DCP upgrade. I've found in error.log
that we have some "Weird vbucket_move_done
" for a proximic2 bucket
, which isn't used for a while now.
So I've deleted this bucket and try to rebalance but it failed. It was few ours ago and rebalance keeps failing:
Rebalance exited with reason {buckets_shutdown_wait_failed,
[{'[email protected]',
{'EXIT',
{old_buckets_shutdown_wait_failed,
["proximic2"]}}},
{'[email protected]',
{'EXIT',
{old_buckets_shutdown_wait_failed,
["proximic2"]}}},
{'[email protected]',
{'EXIT',
{old_buckets_shutdown_wait_failed,
["proximic2"]}}},
{'[email protected]',
{'EXIT',
{old_buckets_shutdown_wait_failed,
["proximic2"]}}}]}
ns_orchestrator002 [email protected] 20:42:59 - Fri Jun 19, 2015
Failed to wait deletion of some buckets on some nodes: [{'[email protected]',
{'EXIT',
{old_buckets_shutdown_wait_failed,
["proximic2"]}}},
{'[email protected]',
{'EXIT',
{old_buckets_shutdown_wait_failed,
["proximic2"]}}},
{'[email protected]',
{'EXIT',
{old_buckets_shutdown_wait_failed,
["proximic2"]}}},
{'[email protected]',
{'EXIT',
{old_buckets_shutdown_wait_failed,
["proximic2"]}}}]
I'm a bit blocked right now. I've tried to create this proximic2
bucket, but I've got information that it already exists. However this bucket is not visible in web GUI or cli. Couchbase runs on debians 7.8. All nodes are in version 3.0.3.
Marek